Sort file using pipe in perl - perl

I am looking for a way to sort a file using pipe. I have checked different examples online but I am still confused
Let's say I have a file called "perlRocks.txt" with different names I want to sort.
This is what I have so far:
open(SORT, "| sort>perlRocks.txt") or die "Can't sort";
close (SORT);
What am I missing?

This isn't using perl to sort. To do this in perl, you would want to:
open ( my $input_fh, "<", "perlRocks.txt" ) or die $!;
my #lines = <$input_fh>;
print sort #lines;
What you're doing is trying to call the command sort.

There is no need to use a pipe, use system instead:
system("sort perlRocks.txt");
This will invoke the system command sort and give it perlRocks.txt as parameter. You will see the output of sort in the shell from which you invoked your script.
Of course, with just this command, the sorted content will be shown and then be forgotten. This might or might not what you have in mind. If you want to permanently store the sorted lines you need to redirect the output into another file.
Of course, perl comes with its own sort operator, so that you don't have to use an external sort command: sort #lines. In order to get the content of your file into #lines, you might want to use the module File::Slurp:
use warnings;
use strict;
use File::Slurp;
my #lines = read_file('perlRocks.txt');
print sort #lines;

You are opening pipe for writing. If you already have this file you probably want read content sorted. See the example with $in below. If you want to write something from your script instead see example with $out below. See open documentation for more variants. For sorting existing file, you have to write to a new file and then rename. It's best to use some shell for this task.
use strict;
use warnings;
use autodie;
use constant FILE_NAME_IN => 'perlRocks.in';
use constant FILE_NAME_OUT => 'perlRocks.out';
open my $in, '-|', 'sort', FILE_NAME_IN;
while (<$in>) {print};
open my $out, '|-', "sort >'#{[FILE_NAME_OUT]}'";
print $out $_, "\n" for qw(foo bar baz quux);
There is safer version for output pipe which avoids problems with shell interpretation of FILE_NAME_OUT content (You can escape this content but ... no.)
open my $out, '|-' or do {
close STDOUT;
open STDOUT, '>', FILE_NAME_OUT;
exec 'sort' or die $!;
};
If you insist you don't want to use shell, you can use Perl.
use strict;
use warnings;
use autodie;
use constant FILE_NAME_IN => 'perlRocks.txt';
use constant FILE_NAME_OUT => 'perlRocks.txt';
# don't bother with fork if you don't want continue with Perl process
my $pid = fork;
die "cannot fork" unless defined $pid;
unless ($pid) { # use just this code inside in this case
close STDIN;
close STDOUT;
open STDIN, '<', FILE_NAME_IN;
unlink FILE_NAME_IN if FILE_NAME_IN eq FILE_NAME_OUT;
open STDOUT, '>', FILE_NAME_OUT;
exec 'sort' or die $!;
}
waitpid( $pid, 0 );
Note FILE_NAME_IN and FILE_NAME_OUT can be same, but it is not in-place sorting anyway. There are both versions of the file at the disc in some time even one can be hidden and inaccessible. There is also good IPC::Run module for this sort of tasks.

Related

Perl script to parse a text file and match a string

I'm editing my question to add more details
The script executes the command and redirects the output to a text file.
The script then parses the text file to match the following string " Standard 1.1.1.1"
The output in the text file is :
Host Configuration
------------------
Profile Hostname
-------- ---------
standard 1.1.1.1
standard 1.1.1.2
The code works if i search for either 1.1.1.1 or standard . When i search for standard 1.1.1.1 together the below script fails.
this is the error that i get "Unable to find string: standard 172.25.44.241 at testtest.pl
#!/usr/bin/perl
use Net::SSH::Expect;
use strict;
use warnings;
use autodie;
open (HOSTRULES, ">hostrules.txt") || die "could not open output file";
my $hos = $ssh->exec(" I typed the command here ");
print HOSTRULES ($hos);
close(HOSTRULES);
sub find_string
{
my ($file, $string) = #_;
open my $fh, '<', $file;
while (<$fh>) {
return 1 if /\Q$string/;
}
die "Unable to find string: $string";
}
find_string('hostrules.txt', 'standard 1.1.1.1');
Perhaps write a function:
use strict;
use warnings;
use autodie;
sub find_string {
my ($file, $string) = #_;
open my $fh, '<', $file;
while (<$fh>) {
return 1 if /\Q$string/;
}
die "Unable to find string: $string";
}
find_string('output.txt', 'object-cache enabled');
Or just slurp the entire file:
use strict;
use warnings;
use autodie;
my $data = do {
open my $fh, '<', 'output.txt';
local $/;
<$fh>;
};
die "Unable to find string" if $data !~ /object-cache enabled/;
You're scanning a file for a particular string. If that string is not found in that file, you want an error thrown. Sounds like a job for grep.
use strict;
use warnings;
use features qw(say);
use autodie;
use constant {
OUTPUT_FILE => 'output.txt',
NEEDED_STRING => "object-cache enabled",
};
open my $out_fh, "<", OUTPUT_FILE;
my #output_lines = <$out_fh>;
close $out_fh;
chomp #output_lines;
grep { /#{[NEEDED_STRING]}/ } #output_lines or
die qq(ERROR! ERROR! ERROR!); #Or whatever you want
The die command will end the program and exit with a non-zero exit code. The error will be printed on STDERR.
I don't know why, but using qr(object-cache enabled), and then grep { NEEDED_STRING } didn't seem to work. Using #{[...]} allows you to interpolate constants.
Instead of constants, you might want to be able to pass in the error string and the name of the file using GetOptions.
I used the old fashion <...> file handling instead of IO::File, but that's because I'm an old fogy who learned Perl back in the 20th century before it was cool. You can use IO::File which is probably better and more modern.
ADDENDUM
Any reason for slurping the entire file in memory? - Leonardo Herrera
As long as the file is reasonably sized (say 100,000 lines or so), reading the entire file into memory shouldn't be that bad. However, you could use a loop:
use strict;
use warnings;
use features qw(say);
use autodie;
use constant {
OUTPUT_FILE => 'output.txt',
NEEDED_STRING => qr(object-cache enabled),
};
open my $out_fh, "<", OUTPUT_FILE;
my $output_string_found; # Flag to see if output string is found
while ( my $line = <$out_fh> ) {
if ( $line =~ NEEDED_STRING ){
$output_string_found = "Yup!"
last; # We found the string. No more looping.
}
}
die qq(ERROR, ERROR, ERROR) unless $output_string_found;
This will work with the constant NEEDED_STRING defined as a quoted regexp.
perl -ne '/object-cache enabled/ and $found++; END{ print "Object cache disabled\n" unless $found}' < input_file
This just reads the file a line at a time; if we find the key phrase, we increment $found. At the end, after we've read the whole file, we print the message unless we found the phrase.
If the message is insufficient, you can exit 1 unless $found instead.
I suggest this because there are two things to learn from this:
Perl provides good tools for doing basic filtering and data munging right at the command line.
Sometimes a simpler approach gets a solution out better and faster.
This absolutely isn't the perfect solution for every possible data extraction problem, but for this particular one, it's just what you need.
The -ne option flags tell Perl to set up a while loop to read all of standard input a line at a time, and to take any code following it and run it into the middle of that loop, resulting in a 'run this pattern match on each line in the file' program in a single command line.
END blocks can occur anywhere and are always run at the end of the program only, so defining it inside the while loop generated by -n is perfectly fine. When the program runs out of lines, we fall out the bottom of the while loop and run out of program, so Perl ends the program, triggering the execution of the END block to print (or not) the warning.
If the file you are searching contained a string that indicated the cache was disabled (the condition you want to catch), you could go even shorter:
perl -ne '/object-cache disabled/ and die "Object cache disabled\n"' < input_file
The program would scan the file only until it saw the indication that the cache was disabled, and would exit abnormally at that point.
First, why are you using Net::SSH::Expect? Are you executing a remote command? If not, all you need to execute a program and wait for its completion is system.
system("cmd > file.txt") or die "Couldn't execute: $!";
Second, it appears that what fails is your regular expression. You are searching for the literal expression standard 1.1.1.1 but in your sample text it appears that the wanted string contains either tabs or several spaces instead of a single space. Try changing your call to your find_string function:
find_string('hostrules.txt', 'standard\s+1.1.1.1'); # note '\s+' here

writing a script in perl to convert all files in a folder to another format

Question is 2 fold:
I'm writing a perl (not to experienced with perl) script and can get it to convert one file at a time from csv to ascii. I want to do a loop that takes all csv's in a folder and converts them to ascii/txt.
Is perl the best language to be attempting this with? I assumed yes since i can successfully do it one file at a time but having a very hard time figuring out a way to loop it.
I was trying to figure out how to load all the files into an array then run the loop for each one, but my googling has reached its limit and i'm out of ideas.
here's my working script:
#!/usr/bin/env perl
use strict;
use warnings;
use autodie;
use open IN => ':encoding(UTF-16)';
use open OUT => ':encoding(ascii)';
my $buffer;
open(my $ifh, '<', 'Software_compname.csv');
read($ifh, $buffer, -s $ifh);
close($ifh);
open(my $ofh, '>', 'Software_compname.txt');
print($ofh $buffer);
close($ofh);
Just add the following loop to your script and give it the files to process as arguments:
for my $input_file (glob shift) {
(my $output_file = $input_file) =~ s/csv$/txt/ or do {
warn "Invalid file name: $input_file\n";
next;
};
my $buffer;
open my $ifh, '<', $input_file;
read $ifh, $buffer, -s $ifh;
close $ifh;
open my $ofh, '>', $output_file;
print{$ofh} $buffer;
close $ofh;
}
If you want to do it from the Perl side only, I would suggest using File::Find::Rule:
use File::Find::Rule;
my #files = File::Find::Rule->file()
->name('*.in')
->in(my #input_directories = ('.'));
# etc.
regards,
matthias

Why is my Perl program not reading from the input file?

I'm trying to read in this file:
Oranges
Apples
Bananas
Mangos
using this:
open (FL, "fruits");
#fruits
while(<FL>){
chomp($_);
push(#fruits,$_);
}
print #fruits;
But I'm not getting any output. What am I missing here? I'm trying to store all the lines in the file into an array, and printing out all the contents on a single line. Why isn't chomp removing the newlines from the file, like it's supposed to?
you should always use :
use strict;
use warnings;
at the begining of your scripts.
and use 3 args open, lexical handles and test opening for failure, so your script becomes:
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
my #fruits;
my $file = 'fruits';
open my $fh, '<', $file or die "unable to open '$file' for reading :$!";
while(my $line = <$fh>){
chomp($line);
push #fruits, $line;
}
print Dumper \#fruits;
I'm guessing that you have DOS-style newlines (i.e., \r\n) in your fruits file. The chomp command normally only works on unix-style (i.e., \n.)
You're not opening any file. FL is a file handle that never is opened, and therefore you can't read from it.
The first thing you need to do is put use warnings at the top of your program to help you with these problems.
#!/usr/bin/env perl
use strict;
use warnings;
use IO::File;
use Data::Dumper;
my $fh = IO::File->new('fruits', 'r') or die "$!\n";
my #fruits = grep {s/\n//} $fh->getlines;
print Dumper \#fruits;
that's nice and clean
You should check open for errors:
open( my $FL, '<', 'fruits' ) or die $!;
while(<$FL>) {
...
1) You should always print the errors from IO. `open() or die "Can't open file $f, $!";
2) you probably started the program from different directory from where file "fruits" is

What's the easiest way to write to a file using Perl?

Currently I am using
system("echo $panel_login $panel_password $root_name $root_pass $port $panel_type >> /home/shared/ftp");
What is the easiest way to do the same thing using Perl? IE: a one-liner.
Why does it need to be one line? You're not paying by the line, are you? This is probably too verbose, but it took a total of two minutes to type it out.
#!/usr/bin/env perl
use strict;
use warnings;
my #values = qw/user secret-password ftp-address/;
open my $fh, '>>', 'ftp-stuff' # Three argument form of open; lexical filehandle
or die "Can't open [ftp-stuff]: $!"; # Always check that the open call worked
print $fh "#values\n"; # Quote the array and you get spaces between items for free
close $fh or die "Can't close [ftp-stuff]: $!";
You might find IO::All to be helpful:
use IO::All;
#stuff happens to set the variables
io("/home/shared/ftp")->write("$panel_login $panel_password $root_name $root_pass $port $panel_type");
EDIT (By popular and editable demand)
http://perldoc.perl.org/functions/open.html
In your case you would have to :
#21st century perl.
my $handle;
open ($handle,'>>','/home/shared/ftp') or die("Cant open /home/shared/ftp");
print $handle "$panel_login $panel_password $root_name $root_pass $port $panel_type";
close ($handle) or die ("Unable to close /home/shared/ftp");
Alternatively, you could use the autodie pragma (as #Chas Owens suggested in comments).
This way, no check (the or die(...)) part needs to be used.
Hope to get it right this time. If so, will erase this Warning.
Old deprecated way
Use print (not one liner though). Just open your file before and get a handle.
open (MYFILE,'>>/home/shared/ftp');
print MYFILE "$panel_login $panel_password $root_name $root_pass $port $panel_type";
close (MYFILE);
http://perl.about.com/od/perltutorials/a/readwritefiles_2.htm
You might want to use the simple File::Slurp module:
use File::Slurp;
append_file("/home/shared/ftp",
"$panel_login $panel_password $root_name $root_pass ".
"$port $panel_type\n");
It's not a core module though, so you'll have to install it.
(open my $FH, ">", "${filename}" and print $FH "Hello World" and close $FH)
or die ("Couldn't output to file: ${filename}: $!\n");
Of course, it's impossible to do proper error checking in a one liner... That should be written slightly differently:
open my $FH, ">", "${filename}" or die("Can't open file: ${filename}: $!\n");
print $FH "Hello World";
close $FH;
For advanced one-liners like this, you could also use the psh command from Psh, a simple pure Perl shell.
psh -c '{my $var = "something"; print $var} >/tmp/out.txt'
I use FileHandle. From the POD:
use FileHandle;
$fh = new FileHandle ">> FOO"; # modified slightly from the POD, to append
if (defined $fh) {
print $fh "bar\n";
$fh->close;
}
If you want something closer to a "one-liner," you can do this:
use FileHandle;
my $fh = FileHandle->new( '>> FOO' ) || die $!;
$fh->print( "bar\n" );
## $fh closes when it goes out of scope
You can do a one-liner like this one:
print "$panel_login $panel_password $root_name $root_pass $port $panel_type" >> io('/home/shared/ftp');
You only need to add the IO::All module to your code, like this:
use IO::All;
Some good reading about editing files with perl:
FMTYEWTK About Mass Edits In Perl

How can I change the standard output filehandle for a system call in Perl?

I am trying change the stdout of the system function using select. But it does not seem to be working. The system output is getting displayed on console rather than getting redirected to the file.
use strict;
use warnings;
chdir "C:\\Documents and Settings\\" or die "cannot open the dir\n";
open FH, ">out.txt" or die "cannot open out.txt\n";
select FH or die " cannot change the stdout\n";
system "dir /a" ;
Although I can do system "dir /a > out.txt", I want to know why the above code is not working.
The select function changes the default filehandle for output. It does not change STDOUT, which points to whatever it points to regardless if it is the default filehandle for output. If you want to change STDOUT, you have to do something like ylebre or Jon's answer.
When you start a child process, it gets the same standard output as the parent. It doesn't care what the default filehandle of the parent is though.
The problem with your code is that system command doesn't capture the output for you. You can use qx//:
use strict;
use warnings;
open my $fh, ">", "output.txt" or die $!;
my $oldfh= select $fh;
print qx/ls -a/;
select $oldfh;
close $fh or warn $!;
There are other options to run an external application and obtain read it's output, like the IPC::Open2 and IPC::Open3 modules. They're included in perl by default.
A side note: don't use globs for file handles: they are globals, and globals are evil. Use lexical variables instead (FH vs. $fh). Also, you should use the three argument open instead of two parameters, since the former is safer than the latter.
Maybe this helps
I'm not sure 'select' is what you are looking for in this case. To redirect STDOUT to a file, you can do the following:
use strict;
use warnings;
use IO::Handle;
open FH, "> out.txt";
STDOUT->fdopen( \*FH, 'w' ) or die $!;
print "Hello world!\n";
system "dir /a";
Hope this helps!
I reassign the file descriptors at the system (not Perl) level:
sub reopen($$$) {
open $_[0], $_[1]. '&='. fileno($_[2]) or die "failed to reopen: $!";
}
reopen *STDOUT, '>', $new_stdout_handle;
Now anything that you fork will think $new_stdout_handle is standard output.
I pasted a full example to gist: http://gist.github.com/129407