How to read the text file using perl command line arguments and print the third column using perl?
I'm struck with taking input from the command line and printing the required column. Help me to choose the right way to reach the expected output.
Code which I wrote to take command line input:(map.pl)
use strict;
use warnings 'all';
use Getopt::Long 'GetOptions';
my #files=GetOptions(
'p_file=s' => \my $p_file,
);
print $p_file ? "p_file = $p_file\n" : "p_file\n";
Output I got for above code:
perl map.pl -p_file cat.txt
p_file = cat.txt
cat.txt:(Input file)
ADG:YUF:TGH
UIY:POG:YTH
GHJUR:"HJKL:GHKIO
Expected output:
TGH
YTH
GHKIO
Perl can automatically read files whose names are provided as command line arguments. The command below should produce your expected output
perl -F: -le 'print $F[2]' cat.txt
-F: turns on autosplit mode, sets the field separator to : and loops over lines of input files. -l handles line endings during input and output. The code after e flag ('print $F[2]' prints 3rd field) is executed for each line of file. Find out more by reading perldoc perlrun.
You'd need to read the file and split the lines to get the columns, and print the required column. Here's a demo code snippet, using the perl -s switch to parse command line arguments. Run like this ./map.pl -p_file=cat.txt
#!/usr/bin/perl -s
use strict;
use warnings;
use vars qw[$p_file];
die("You need to pass a filename as argument") unless(defined($p_file));
die("Filename ($p_file) does not exists") unless(-f $p_file);
print "Proceeding to read file : $p_file\n\n";
open(my $fh,'<',$p_file) or die($!);
while(chomp(my $line = <$fh>)) {
next unless(defined($line) && $line);
my #cols = split(/:/,$line);
print $cols[-1],"\n";
}
close($fh);
Instead of printing the file into one line, the output is a bunch of new lines. What am I ding wrong in the file path of this perl script?
Script:
my $app = $ARGV[0];
my $day=`date -v-1d '+%d'`;
my $month=`date -v-1d '+%b'`;
my $yr=`date -v-1d '+%Y'`;
my $file = "/path/to/file/$app/$yr/$month/treshold-$day .log";
print $file;
Result:
$ perl test.pl inter
/path/to/file/inter/2013
/Dec
/treshold-13
Output should be: /path/to/file/inter/2013/Dec/treshold-13.log
date(1) write an end-of-line at the end of the output. Use chomp to throw it away.
I have some text files, inside a directory and i want to parse their content and write it to a file. So far the code i am using is this:
#!/usr/bin/perl
#The while loop repeats the execution of a block as long as a certain condition is evaluated true
use strict; # Always!
use warnings; # Always!
my $header = 1; # Flag to tell us to print the header
while (<*.txt>) { # read a line from a file
if ($header) {
# This is the first line, print the name of the file
**print "========= $ARGV ========\n";**
# reset the flag to a false value
$header = undef;
}
# Print out what we just read in
print;
}
continue { # This happens before the next iteration of the loop
# Check if we finished the previous file
$header = 1 if eof;
}
When i run this script i am only getting the headers of the files, plus a compiled.txt entry.
I also receive the following message in cmd : use of uninitialized $ARGV in concatenation <.> or string at concat.pl line 12
So i guess i am doing something wrong and $ARGV isn't used at all. Plus instead of $header i should use something else in order to retrieve the text.
Need some assistance!
<*.txt> does not read a line from a file, even if you say so in a comment. It runs
glob '*.txt'
i.e. the while loop iterates over the file names, not over their contents. Use empty <> to iterate over all the files.
BTW, instead of $header = undef, you can use undef $header.
As I understand you want to print a header with the filename just before the first line, and concatenate them all to a new one. Then a one-liner could be enough for the task.
It checks first line with variable $. and closes the filehandle to reset its value between different input files:
perl -pe 'printf qq|=== %s ===\n|, $ARGV if $. == 1; close ARGV if eof' *.txt
An example in my machine yields:
=== file1.txt ===
one
=== file2.txt ===
one
two
Ok, so I'm very new to Perl. I have a text file and in the file there are 4 columns of data(date, time, size of files, files). I need to create a small script that can open the file and get the average size of the files. I've read so much online, but I still can't figure out how to do it. This is what I have so far, but I'm not sure if I'm even close to doing this correctly.
#!/usr/bin/perl
open FILE, "files.txt";
##array = File;
while(FILE){
#chomp;
($date, $time, $numbers, $type) = split(/ /,<FILE>);
$total += $numbers;
}
print"the total is $total\n";
This is how the data looks in the file. These are just a few of them. I need to get the numbers in the third column.
12/02/2002 12:16 AM 86016 a2p.exe
10/10/2004 11:33 AM 393 avgfsznew.pl
11/01/2003 04:42 PM 38124 c2ph.bat
Your program is reasonably close to working. With these changes it will do exactly what you want
Always use use strict and use warnings at the start of your program, and declare all of your variables using my. That will help you by finding many simple errors that you may otherwise overlook
Use lexical file handles, the three-parameter form of open, and always check the return status of any open call
Declare the $total variable outside the loop. Declaring it inside the loop means it will be created and destroyed each time around the loop and it won't be able to accumulate a total
Declare a $count variable in the same way. You will need it to calculate the average
Using while (FILE) {...} just tests that FILE is true. You need to read from it instead, so you must use the readline operator like <FILE>
You want the default call to split (without any parameters) which will return all the non-space fields in $_ as a list
You need to add a variable in the assignment to allow for athe AM or PM field in each line
Here is a modification of your code that works fine
use strict;
use warnings;
open my $fh, '<', "files.txt" or die $!;
my $total = 0;
my $count = 0;
while (<$fh>) {
my ($date, $time, $ampm, $numbers, $type) = split;
$total += $numbers;
$count += 1;
}
print "The total is $total\n";
print "The count is $count\n";
print "The average is ", $total / $count, "\n";
output
The total is 124533
The count is 3
The average is 41511
It's tempting to use Perl's awk-like auto-split option. There are 5 columns; three containing date and time information, then the size and then the name.
The first version of the script that I wrote is also the most verbose:
perl -n -a -e '$total += $F[3]; $num++; END { printf "%12.2f\n", $total / ($num + 0.0); }'
The -a (auto-split) option splits a line up on white space into the array #F. Combined with the -n option (which makes Perl run in a loop that reads the file name arguments in turn, or standard input, without printing each line), the code adds $F[3] (the fourth column, counting from 0) to $total, which is automagically initialized to zero on first use. It also counts the lines in $num. The END block is executed when all the input is read; it uses printf() to format the value. The + 0.0 ensures that the arithmetic is done in floating point, not integer arithmetic. This is very similar to the awk script:
awk '{ total += $4 } END { print total / NR }'
First drafts of programs are seldom optimal — or, at least, I'm not that good a programmer. Revisions help.
Perl was designed, in part, as an awk killer. There is still a program a2p distributed with Perl for converting awk scripts to Perl (and there's also s2p for converting sed scripts to Perl). And Perl does have an automatic (built-in) variable that keeps track of the number of lines read. It has several names. The tersest is $.; the mnemonic name $NR is available if you use English; in the script; so is $INPUT_LINE_NUMBER. So, using $num is not necessary. It also turns out that Perl does a floating point division anyway, so the + 0.0 part was unnecessary. This leads to the next versions:
perl -MEnglish -n -a -e '$total += $F[3]; END { printf "%12.2f\n", $total / $NR; }'
or:
perl -n -a -e '$total += $F[3]; END { printf "%12.2f\n", $total / $.; }'
You can tune the print format to suit your whims and fancies. This is essentially the script I'd use in the long term; it is fairly clear without being long-winded in any way. The script could be split over multiple lines if you desired. It is a simple enough task that the legibility of the one-line is not a problem, IMNSHO. And the beauty of this is that you don't have to futz around with split and arrays and read loops on your own; Perl does most of that for you. (Granted, it does blow up on empty input; that fix is trivial; see below.)
Recommended version
perl -n -a -e '$total += $F[3]; END { printf "%12.2f\n", $total / $. if $.; }'
The if $. tests whether the number of lines read is zero or not; the printf and division are omitted if $. is zero so the script outputs nothing when given no input.
There is a noble (or ignoble) game called 'Code Golf' that was much played in the early days of Stack Overflow, but Code Golf questions are no longer considered good questions. The object of Code Golf is to write a program that does a particular task in as few characters as possible. You can play Code Golf with this and compress it still further if you're not too worried about the format of the output and you're using at least Perl 5.10:
perl -Mv5.10 -n -a -e '$total += $F[3]; END { say $total / $. if $.; }'
And, clearly, there are a lot of unnecessary spaces and letters in there:
perl -Mv5.10 -nae '$t+=$F[3];END{say$t/$.if$.}'
That is not, however, as clear as the recommended version.
#!/usr/bin/perl
use warnings;
use strict;
open my $file, "<", "files.txt";
my ($total, $cnt);
while(<$file>){
$total += (split(/\s+/, $_))[3];
$cnt++;
}
close $file;
print "number of files: $cnt\n";
print "total size: $total\n";
printf "avg: %.2f\n", $total/$cnt;
Or you can use awk:
awk '{t+=$4} END{print t/NR}' files.txt
Try doing this :
#!/usr/bin/perl -l
use strict; use warnings;
open my $file, '<', "my_file" or die "open error [$!]";
my ($total, $count);
while (<$file>){
chomp;
next if /^$/;
my ($date, $time, $x, $numbers, $type) = split;
$total += $numbers;
$count++;
}
print "the average is " . $total/$count . " and the total is $total";
close $file;
It is as simple as this:
perl -F -lane '$a+=$F[3];END{print "The average size is ".$a/$.}' your_file
tested below:
> cat temp
12/02/2002 12:16 AM 86016 a2p.exe
10/10/2004 11:33 AM 393 avgfsznew.pl
11/01/2003 04:42 PM 38124 c2ph.bat
Now the execution:
> perl -F -lane '$a+=$F[3];END{print "The average size is ".$a/$.}' temp
The average size is 41511
>
explanation:
-F -a says store the line in an array format.with the default separator as space or tab.
so nopw $F[3] has you size of the file.
sum up all the sizes in the 4th column untill all the lines are processed.
END will be executed after processing all the lines in the file.
so $. at the end will gives the number of lines.
so $a/$. will give the average.
This solution opens the file and loops through each line of the file. It then splits the file into the five variables in the line by splitting on 1 or more spaces.
open the file for reading, "<", and if it fails, raise an error or die "..."
my ($total, $cnt) are our column total and number of files added count
while(<FILE>) { ... } loops through each line of the file using the file handle and stores the line in $_
chomp removes the input record separator in $_. In unix, the default separator is a newline \n
split(/\s+/, $_) Splits the current line represented by$_, with the delimiter \s+. \s represents a space, the + afterward means "1 or more". So, we split the next line on 1 or more spaces.
Next we update $total and $cnt
#!/usr/bin/perl
open FILE, "<", "files.txt" or die "Error opening file: $!";
my ($total, $cnt);
while(<FILE>){
chomp;
my ($date, $time, $am_pm, $numbers, $type) = split(/\s+/, $_);
$total += $numbers;
$cnt++;
}
close FILE;
print"the total is $total and count of $cnt\n";`
My test file has "n" number of lines and between each line there is a ^M, which in turn makes it one big string. The code I am working with opens said file and should parse out a header and then the subsequent rows, then searches for the Directory Path and File name. But because the file just ends up as a big string it doesn't work correctly
#!/usr/bin/perl
#use strict;
#use warnings;
open (DATA, "<file.txt") or die ("Unable to open file");
my $search_string = "Directory Path";
my $column_search = "Filename";
my $header = <DATA>;
my #header_titles = split /\t/, $header;
my $extract_col = 0;
my $col_search = 0;
for my $header_line (#header_titles) {
last if $header_line =~ m/$search_string/;
$extract_col++;
}
for my $header_line (#header_titles) {
last if $header_line =~m/$column_search/;
$col_search++;
}
print "Extracting column $extract_col $search_string\n";
while ( my $row = <DATA> ) {
last unless $row =~ /\S/;
chomp $row;
my #cells = split /\t/, $row;
$cells[74]=~s/:/\//g;
$cells[$extract_col]= $cells[74] . $cells[$col_search];
print "$cells[$extract_col] \n";
}
When i open the test file in VI i have used
:%s/^M/\r/g
and that removes the ^M's but how do i do it inside this perl program? When i tried a test program and inserted that s\^M/\r/g and had it write to a different file it came up as a lot of Chinese characters.
If mac2unix isn't working for you, you can write your own mac2unix as a Perl one-liner:
perl -pi -e 'tr/\r/\n/' file.txt
That will likely fail if the size of the file is larger than virtual memory though, as it reads the whole file into memory.
For completeness, let's also have a dos2unix:
perl -pi -e 'tr/\r//d' file.txt
and a unix2dos:
perl -pi -e 's/\n/\r\n/g' file.txt
Before you start reading the file, set $/ to "\r". This is set to the linefeed character by default, which is fine for UNIX-style line endings, and almost OK for DOS-style line endings, but useless for the old Mac-style line endings you are seeing. You can also try mac2unix on your input file if you have it installed.
For more, look for "INPUT_RECORD_SEPARATOR" in the perlvar manpage.
Did this file originate on a windows system? If so, try running the dos2unix command on the file before reading it. You can do this before invoking the perl script or inside the script before you read it.
You might want to set $\ (input record separator) to ^M in the beginning of your script, such as:
$\ = "^M";
perl -MExtUtils::Command -e dos2unix file