Pattern matching in perl & assigning - perl

I need to search for a particular string in a file and then assign it to a variable, example: in the file content it is written as CURRENT_RUN_ID=1636, so I need to search for string CURRENT_RUN_ID and assign its given value i.e. 1636 toa variable sayrunvar`, for this I tried below given, but it doesn't seem to be working, can you correct me here?
opendir(DAPATH,$sDAPATH) or die "Can't open $sDAPATH: $!";
print OUTLOG "\nfound da path : $sDAPATH\n";
my #adirs = readdir(DAPATH);
print OUTLOG "Starting capturing DA\n";
my $da = glob "*$runVar.csv*";
print OUTLOG "Assigned DA";
closedir(DAPATH);
}

It is not matching because you are giving spaces at if (/CURRENT_RUN_ID = \s*(.*)/) in the match.It is searching for spaces in the string.
The pattern what you are trying to match will match CURRENT_RUN_ID = 1636 string, notice the spaces between CURRENT_RUN_ID and = and after =. If there is match then the no of spaces in the pattern should be exactly same as the no of spaces in the string.
There is space between CURRENT_RUN_ID and = and also after =.
Better remove the faulty spaces and make the space optional using \s* try this:
if (/CURRENT_RUN_ID\s*=\s*(.*)/){
my $runvar = $1;
print "$runvar \n";
}
EDIT:
As per your requirement I changed your script as(I am not writing to file):
#!/usr/bin/perl
use strict;
use warnings;
open my $fh, '<', 'file' or die "unable to open file : $! \n";
my $runVar="";
while(<$fh>){
if (/CURRENT_RUN_ID\s*=\s*(.*)/){
print "we can now assign run id\n";
$runVar = $1;
print "assigned current run id to variable\n";
}
else {
print "run id not assigned\n";
}
}
close($fh);

Here you are not matching any of the lines while performing the pattern matching in line 18. And even you can reduce few repetitive steps in the code.
But for your convenience I have used your program.
#!C:\Strawberry\perl\bin
use strict;
use warnings;
my $sSuccessString = "CURRENT_RUN_ID";
open(LOG,"$slogfile") or die("Can't open $slogfile\n");
my $sLines;
{
local $/ = undef;
$sLines=<LOG>;
}
if($sLines =~ m/$sSuccessString/){
open(OUTLOG, ">>test.txt");
print OUTLOG "found current run id in log\n";
print OUTLOG "it is found in log as : $sSuccessString \n";
#if ($sLines =~ m/(CURRENT_RUN_ID=.*)/i) {
#print OUTLOG "<p>" . $1 . "<\/p>\n";
#In below line you need to match the pattern with the line.
if ($sLines=~/CURRENT_RUN_ID=(.*)/){
print OUTLOG "we can now assign run id\n";
my $runVar = $1;
print "$runVar\n";
print OUTLOG "assigned current run id to variable\n";
}
else {
print "run id not assigned\n";
}
}
I have copied your log file content to one of my text file (prog.txt) on my desktop and ran the script. Please see the output.
#!C:\Strawberry\perl\bin
use strict;
use warnings;
my $sSuccessString = "CURRENT_RUN_ID";
open(LOG,"Prog.txt") or die("Can't open text file\n");
my $sLines;
{
local $/ = undef;
$sLines=<LOG>;
}
if($sLines =~ m/$sSuccessString/){
open(OUTLOG, ">>test.txt");
print OUTLOG "found current run id in log\n";
print OUTLOG "it is found in log as : $sSuccessString \n";
#if ($sLines =~ m/(CURRENT_RUN_ID=.*)/i) {
#print OUTLOG "<p>" . $1 . "<\/p>\n";
if ($sLines=~/CURRENT_RUN_ID=(.*)/){
print OUTLOG "we can now assign run id\n";
my $runVar = $1;
print "$runVar\n";
print OUTLOG "assigned current run id to variable\n";
}
else {
print "run id not assigned\n";
}
}
OUTPUT:
C:\Users\hclabv\Desktop>perl run1.pl
1637
And my log file (test.txt) where we are capturing the logs while running script is as below.
found current run id in log
it is found in log as : CURRENT_RUN_ID
we can now assign run id
assigned current run id to variable
For getting runVar.csv file from desired directory, I have added the below piece of code with in the if loop
opendir(DAPATH,"C:/Users/hclabv/Desktop/Scripting/files") or die "Can't open DAPATH: $!";
print OUTLOG "\nfound da path : DAPATH\n";
my #adirs = readdir(DAPATH);
print "csv files #adirs\n";
print OUTLOG "Starting capturing DA\n";
my $csvfile = "$runVar.csv";
foreach my $adirs (#adirs) {
if ($adirs eq $csvfile) {
print "file found\n";
my $da = $adirs;
print "csv file is $da\n";
}
print OUTLOG "Assigned DA";
close(DAPATH);
}

Related

Arguments with quotations(" ") to be not escaped

I am writing a Perl script which enables the addition and modification of parameters maintained in a particular file.
The script takes the following arguments; Parameter name($paraName), Parameter value($paraVal) and the file ($profile).
The script checks if the parameter($paraName) exists already. if it does, it just changes the value($paraVal) else adds both the parameter($paraName) and the value($paraVal) to the file($profile).
Following is the block of code for the same:
print " checking if parameter is already avaialable";
my $response = system("egrep -qs \"$paraName =\" $profile");
$rc = 1;
if ($response == 0) {
print " Parameter is already available, changing the value now! ";
$rc = system("sed -i 's:.*$paraName.*:$paraName = $paraVal \# Parameter changed by $script:' $profile");
print " Parameter $paraName has been updated with the value $paraVal in the Profile successfully \n\n";
}
else{
print " Parameter is not available, Adding the Paremeter now! ";
$rc = system("echo \"$paraName = $paraVal \# Parameter added by $script\" >> $profile");
print " Parameter $paraName has been added with the value $paraVal in the Profile successfully \n\n";
}
The script works fine for most cases, except when I have arguments with double quotes to be added as a new parameter. It works file for hash(#), slashes (), etc, when passes within a single quote(' ').
This is working in case of changing the value($paraVal) when the parameter($paraName) already exists. But while a new parameter has to be added, this fails to add double quotes in the parameter name.
Would appreciate some help here.
Here is an example of how you could write it as pure Perl:
use feature qw(say);
use strict;
use warnings;
my ( $paraName, $paraVal, $profile ) = #ARGV;
my $script = $0;
open ( my $fh, '<', $profile ) or die "Could not open file '$profile': $!";
my $found = 0;
while( my $line = <$fh> ) {
chomp $line;
if ( my ($key) = $line =~ /^(\Q$paraName\E)\s*=\s*/) {
say "$key = $paraVal \# Parameter changed by $script";
$found = 1;
}
else {
say $line;
}
}
close $fh;
if ( !$found ) {
say "$paraName = $paraVal \# Parameter added by $script";
}
Edit:
The above script does not modify the profile file, but instead writes the modified file to standard output. So it was meant to be used together with Shell redirection to save the output to a new file. To modify the profile file directly, you could use:
use feature qw(say);
use strict;
use warnings;
die "Bad arguments!" if #ARGV != 3;
my ( $paraName, $paraVal, $profile ) = #ARGV;
my $script = $0;
#ARGV = ( $profile );
$^I = '.bak';
my $found = 0;
while (my $line = <<>>) {
chomp $line;
if ( my ($key) = $line =~ /^(\Q$paraName\E)\s*=\s*/) {
say "$key = $paraVal \# Parameter changed by $script";
$found = 1;
}
else {
say $line;
}
} continue {
say "$paraName = $paraVal \# Parameter added by $script" if eof && !$found;
}
This will first save the original profile file to a backup file with .bak extension, and then overwrite the profile file with the modified content.
Try following code as alternative
#!/usr/bin/perl
use strict;
use warnings;
use feature 'say';
use Pod::Usage;
use Getopt::Long;
use Data::Dumper;
my %opt; # program options
my %param; # parameters storage
my $fh; # file handle
GetOptions (
'file|f=s' => \$opt{file},
'name|n=s' => \$opt{name},
'value|v=s' => \$opt{value},
'operation|o=s' => \$opt{op},
'help|h' => \$opt{help},
'man|m' => \$opt{man},
'debug|d' => \$opt{debug}
) or pod2usage(1);
pod2usage(1) if $opt{help};
pod2usage(-exitval => 0, -versose => 2) if $opt{man};
pod2usage(1) unless $opt{file};
open $fh, "< $opt{file}"
or die "Couldn't open $opt{file}";
my #lines = <$fh>;
close $fh;
chomp #lines;
print Dumper(\#lines) if $opt{debug};
push #lines, "$opt{name} = $opt{value}"
if $opt{op} eq 'add';
#lines = map { /$opt{name}\s*=/ ? '' : $_ } #lines
if $opt{op} eq 'del';
#lines = map {
s/($opt{name})\s*=\s*(.*)/$1 = $opt{value}/; $_
} #lines if $opt{op} eq 'mod';
map{ say } #lines
if $opt{op} eq 'view';
map {
/$opt{name}\s*=\s*(.*)/ and say 'Verify: '
. ($1 eq $opt{value} ? 'ok' : 'no')
} #lines if $opt{op} eq 'check';
my %save = map { $_ => 1 } qw/add del mod/;
print Dumper(\#lines) if $opt{debug};
if( $save{ $opt{op} } ) {
open $fh, "> $opt{file}"
or die "Couldn't open $opt{file}";
map { say $fh $_ } #lines;
close $fh;
}
__END__
=head1 NAME
program - modify configuration file
=head1 SYNOPSIS
program [options] [file ...]
Usage:
program -op [add|del|mod|view|check] -n param -v value -f file
Options:
--file,-f configuration filename
--name,-n parameter name
--value,-v parameter value
--operation,-o operation to perform
--help,-h brief help message
--man,-m full documentation
--debug,-d debug mode
=head1 OPTIONS
=over 8
=item B<--file,-f>
Configuration file to edit
=item B<--name,-n>
Configuration parameter name to operate on
=item B<--value,-v>
Configuration parameter value to operate on
=item B<--operation,-o>
Operation to perform on parameter: add, del, mod, view, check
=item B<--debug,-d>
Debug flag to print debug messages.
=item B<--help,-h>
Print a brief help message and exits.
=item B<--man,-m>
Prints the manual page and exits.
=back
=head1 DESCRIPTION
B<This program> allows to operate on configuation files variables.
=head1 AUTHOR
B<Polar Bear> L<https://stackoverflow.com/users/12313309/polar-bear>
=cut

Picking a specific line with a specific string

I am trying this in Perl to pick one complete line from whole document which contains "CURRENT_RUN_ID". I have been using below code to accomplish the above said task but I am unable to enter the while loop.
my $sSuccessString = "CURRENT_RUN_ID";
open(LOG, "$slogfile") or die("Can't open $slogfile\n");
my $sLines;
{
local $/ = undef;
$sLines=<LOG>;
}
my $spool = 0;
my #matchingLines;
while (<LOG>)
{
print OUTLOG "in while loop\n";
if (m/$sSuccessString/i) {
print OUTLOG "in if loop\n";
$spool = 1;
print map { "$_ \n" } #matchingLines;
#matchingLines = ();
}
if ($spool) {
push (#matchingLines, $_);
}
}
You are already done reading from the filehandle LOG after you have slurped it into $sLines. <LOG> in the head of the while will return undef because it has reached eof. You either have to use that variable $sLines in your while loop or get rid of it. You're not using it anyway.
If you only want to print the line that matches, all you need to do is this:
use strict;
use warnings;
open my $fh_in, '<', 'input_file' or die $!;
open my $fh_out '>', 'output_file' or die $!;
while (my $line = <$fh_in>) {
print $fh_out $line if $line =~ m/CURRENT_RUN_ID/;
}
close $fh_in;
close $fh_out;
When you execute this code:
$sLines=<LOG>;
it reads all of the data from LOG into $sLines and it leaves the file pointer for LOG at the end of the file. So when you next try to read from that file handle with:
while (<LOG>)
nothing is returned as there is no more data to read.
If you want to read the file twice, then you will need to use the seek() function to reset the file pointer before your second read.
seek LOG, 0, 0;
But, given that you never do anything with $sLines I suspect that you can probably just remove that whole section of the code.
The whole thing with $spool and #matchingLines seems strange too. What were you trying to achieve there?
I think your code can be simplified to just:
my $sSuccessString = "CURRENT_RUN_ID";
open(LOG, $slogfile) or die("Can't open $slogfile\n");
while (<LOG>) {
print OUTLOG if /$sSuccessString/i/;
}
Personally, I'd make it even simpler, by reading from STDIN and writing to STDOUT.
my $sSuccessString = 'CURRENT_RUN_ID';
while (<>) {
print if /$sSuccessString/i/;
}
And then using Unix I/O redirection to connect up the correct files.
$ ./this_filter.pl < your_input.log > your_output.log

Perl - Using a Variable that is inside an if statement

I've been putting together a Perl script that defines a variable, and then via an if statement will assign a new value to the variable.
I want to use the last assigned value from the if statement and reference it somewhere else outside the if altogether.
It appears that when referencing it that it uses the original value.
Here's the code:
## VARIABLES
my $FILENAME = $input[0];
open my $info, $DATAFILE or die "can't open <$DATAFILE> for reading $!";
{
while (my $line = <$info>) {
chomp $line;
my #input = split(':', $line);
chomp(#input);
$FILENAME = $input[0];
$PERMIT = $input[1];
$FILESIZE = -s "$TIFF_DL_LOCATION/$PERMIT\_$FILENAME";
$SHORT_PERMIT = substr($PERMIT, 0, 2);
### DEBUG ONLY ###
print "$FILENAME / $PERMIT / $FTPBASE/$SHORT_PERMIT/$PERMIT/$FILENAME\n";
my $ftp = Net::FTP::Throttle->new(
"example.com",
MegabitsPerSecond => $THROTTLELVL,
Debug => $DEBUGLVL
) or die "Cannot connect: $#";
$ftp->login("anonymous", 'anonymous') or die "Cannot login ", $ftp->message;
$ftp->binary or die "Unable to set binary mode ", $ftp->message;
if ($PROGRESSBAR eq 1) {
print "\n[$./$LINE_COUNT] Downloading $FILENAME\n";
my $REMOTE_FILESIZE = $ftp->size("/PATH/TO/DATA/$SHORT_PERMIT/$PERMIT/$FILENAME");
if (!defined($REMOTE_FILESIZE)) {
print "=> FILE DOES NOT APPEAR TO EXIST ON FTP SERVER\n";
if ($FILENAME =~ m/_\s/) {
print "=> ATTEMPTING TO FIX NOW\n";
$FILENAME =~ s/_\s/, /g;
$REMOTE_FILESIZE = $ftp->size("/PATH/TO/DATA/$SHORT_PERMIT/$PERMIT/$FILENAME");
if (!defined($REMOTE_FILESIZE)) {
print "=> FAILED!\n";
}
}
elsif ($FILENAME =~ m/_\s\s/) {
print "=> ATTEMPTING TO FIX NOW\n";
$FILENAME =~ s/_\s\s/, /g;
$REMOTE_FILESIZE = $ftp->size("/PATH/TO/DATA/$SHORT_PERMIT/$PERMIT/$FILENAME");
if (!defined($REMOTE_FILESIZE)) {
print "$FILENAME\n";
print "=> FAILED!\n";
}
}
else {
print "=> ALL ATTEMPTS TO RESOLVE THE ISSUE HAVE FAILED.\n";
next;
}
}
$REMOTE_FILESIZE = $ftp->size("/PATH/TO/DATA/$SHORT_PERMIT/$PERMIT/$FILENAME");
print "FILENAME: $FILENAME\n";
--- SNIP SNIP - MORE DATA, NOT RELEVANT--
The output I get is the name of the file that was originally opened, not the value after modification by the substitutions in the if statements.
This is error correction: it checks the filename that it gets from the file, and if it matches something, it corrects it, and in the end, I have an old variable $FILENAME with a new value that I want to use outside the if.
I struggled to understand the code in your question, mainly because of all the upper case strings; both identifiers and comments
Local identifiers in most languages, including Perl, are usually written using lower-case letters, digits, and the underscore _. Capital letters are reserved for global variables, and in Perl's case that is mostly package (class) names
Identifiers of important variables use capital letters so that they stand out from the rest; but in your example most identifiers are all-capitals. It is diffifult to understand the structure of your code if every identifier says it's very important
I have looked carefully at the code you show, and have done my best to reorganise and simplify it so that it is more legible and does what I think you intended. I have added declarations for all of the variables that are not declared within your code sample, and have added what I think are the correct closing braces } to balance the syntax so that it will compile
As others have noted, the second conditional clause that tests for underscore followed by two spaces will never be executed, because the preceding test for underscore followed by one space will already have caught that case. It is also pointless to use a separate pattern match to determine whether a substitution s/// will succeed, as the substitution alone does nothing and returns a false value if there was no match
The intention is to help you to write Perl code that you can understand, as well as others who you may ask for help (like Stack Overflow) or may be given the job of maintaining your software.
use strict;
use warnings;
my ($progressbar, $line_count);
my ($datafile, $ftpbase, $short, $tiff_dl_location);
my ($throttlelvl, $debuglvl);
my #input;
## Variables
my $filename = $input[0];
open my $info, $datafile or die "can't open <$datafile> for reading $!";
{
while (<$info>) {
chomp;
my ($filename, $permit) = split /:/;
my $filesize = -s "$tiff_dl_location/${permit}_${filename}";
my $short_permit = substr $permit, 0, 2;
### DEBUG ONLY ###
print "$filename / $permit / $ftpbase/$short_permit/$permit/$filename\n";
my $file_path = "/PATH/TO/DATA/$short_permit/$permit/$filename";
my $ftp = Net::FTP::Throttle->new(
"example.com",
MegabitsPerSecond => $throttlelvl,
Debug => $debuglvl
) or die "Cannot connect: $#";
$ftp->login(qw/ anonymous anonymous /) or die "Cannot login: ", $ftp->message;
$ftp->binary or die "Unable to set binary mode ", $ftp->message;
if ($progressbar == 1) {
print "\n[$./$line_count] Downloading $filename\n";
my $remote_filesize = $ftp->size($file_path);
if (not defined $remote_filesize) {
warn "=> File does not appear to exist on FTP server\n";
warn "=> Attempting to fix now\n";
$filename =~ s/_\s+/, /g;
$remote_filesize = $ftp->size($file_path);
if (not defined $remote_filesize) {
warn "=> ALL ATTEMPTS TO RESOLVE THE ISSUE HAVE FAILED.\n";
next;
}
}
print "File name: $filename\n";
# --- snip snip - more data, not relevant ---
}
}
}
Declare variables outside of (before) the conditional unless they will only be used inside of that conditional block {}.
Also,
use strict;
use warnings;

Perl Help Needed: Replacing values

I am having an input file like this:
Input file
I need to replace the value #pSBSB_ID="*" of #rectype=#pRECTYPE="SBSB" with #pMEME_SSN="034184233", value of #pRECTYPE="SMSR", ..and have to delete the row where #rectype='#pRECTYPE="SMSR", '
Example:
So, after changes have been made, the file should be like this:
....#pRECTYPE="SBSB", #pGWID="17199269", #pINPUT_METHOD="E", #pGS08="005010X220A1", #pSBSB_FAM_UPDATE_CD="UP", #pSBSB_ID="034184233".....
....#pRECTYPE="SBEL", #pSBEL_EFF_DT="01/01/2013", #pSBEL_UPDATE_CD="TM", #pCSPD_CAT="M", #pCSPI_ID="MHMO1003"
.
.
.
Update
I tried below mentioned code:
Input file extension: mms and there are multiple files to process.
my $save_for_later;
my $record;
my #KwdFiles;
my $r;
my $FilePath = $ARGV[0];
chdir($FilePath);
#KwdFiles = <*>;
foreach $File(#KwdFiles)
{
unless(substr($File,length($File)-4,length($File)) eq '.mms')
{
next;
}
unless(open(INFILE, "$File"))
{
print "Unable to open file: $File";
exit(0);
}
print "Successfully opened the file: \"$File\" for processing\n\n";
while ( my $record = <INFILE> ) {
my %r = $record =~ /\#(\w+) = '(.*?)'/xg;
if ($r{rectype} eq "SMSR") {
$save_for_later = $r{pMEME_SSN};
next;
}
elsif ($r{rectype} eq "SBSB" and $r{pSBSB_ID} eq "*") {
$record =~ s|(\#pSBSB_ID = )'.*?'|$1'$save_for_later'|x;
}
close(INFILE);
}
}
But, I am still not getting the updated values in the file.
#!/usr/bin/perl
open IN, "< in.txt";
open OUT, "> out.txt";
my $CUR_RECID = 1^1;
while (<IN>) {
if ($CUR_RECID) {
s/recname='.+?'/recname='$CUR_RECID'/ if /rectype='DEF'/;
$CUR_RECID = 1^1;
print OUT;
}
$CUR_RECID = $1 if /rectype='ABC'.+?rec_id='(.+?)'/;
}
close OUT;
close IN;
Try that whole code. No need a separate function; This code does everything.
Run this script from your terminal with the files to be modified as arguments:
use strict;
use warnings;
$^I = '.bak'; #modify original file and create a backup of the old ones with .bak appended to the name
my $replacement;
while (<>) {
$replacement = $1 if m/(?<=\#pMEME_SSN=)("\d+")/; #assume replacement will be on the first line of every file.
next if m/^\s*\#pRECTYPE="SMSR"/;
s/(?<=\#pSBSB_ID=)("\*")/$replacement/g;
print;
}

How to write a correct name using combination of variable and string as a filehandler?

I want to make a tool to classify each line in input file to several files
but it seems have some problem in naming a filehandler so I can't go ahead , how do I solve?
here is my program
ARGV[0] is the input file
ARGV[1] is the number of classes
#!/usr/bin/perl
use POSIX;
use warnings;
# open input file
open(Raw,"<","./$ARGV[0]") or die "Can't open $ARGV[0] \n";
# create a directory class to store class files
system("mkdir","Class");
# create files for store class informations
for($i=1;$i<=$ARGV[1];$i++)
{
# it seems something wrong in here
open("Class$i",">","./Class/$i.class") or die "Can't create $i.class \n";
}
# read each line and random decide which class to store
while( eof(Raw) != 1)
{
$Line = readline(*Raw);
$Random_num = ceil(rand $ARGV[1]);
for($k=1;$k<=$ARGV[1];$k++)
{
if($Random_num == $k)
{
# Store to the file
print "Class$k" $Line;
last;
}
}
}
for($h=1;$h<=$ARGV[1];$h++)
{
close "Class$h";
}
close Raw;
thanks
Later I use the advice provided by Bill Ruppert
I put the name of filehandler into array , but it seems appear a syntax bug , but I can't correct it
I label the syntax bug with ######## A syntax error but it looks quite OK ########
here is my code
#!/usr/bin/perl
use POSIX;
use warnings;
use Data::Dumper;
# open input file
open(Raw,"<","./$ARGV[0]") or die "Can't open $ARGV[0] \n";
# create a directory class to store class files
system("mkdir","Class");
# put the name of hilehandler into array
for($i=0;$i<$ARGV[1];$i++)
{
push(#Name,("Class".$i));
}
# create files of classes
for($i=0;$i<=$#Name;$i++)
{
$I = ($i+1);
open($Name[$i],">","./Class/$I.class") or die "Can't create $I.class \n";
}
# read each line and random decide which class to store
while( eof(Raw) != 1)
{
$Line = readline(*Raw);
$Random_num = ceil(rand $ARGV[1]);
for($k=0;$k<=$#Name;$k++)
{
if($Random_num == ($k+1))
{
print $Name[$k] $Line; ######## A syntax error but it looks quite OK ########
last;
}
}
}
for($h=0;$h<=$#Name;$h++)
{
close $Name[$h];
}
close Raw;
thanks
To quote the Perl documentation on the print function:
If you're storing handles in an array or hash, or in general whenever you're using any expression more complex than a bareword handle or a plain, unsubscripted scalar variable to retrieve it, you will have to use a block returning the filehandle value instead, in which case the LIST may not be omitted:
print { $files[$i] } "stuff\n";
print { $OK ? STDOUT : STDERR } "stuff\n";
Thus, print $Name[$k] $Line; needs to be changed to print { $Name[$k] } $Line;.
How about this one:
#! /usr/bin/perl -w
use strict;
use POSIX;
my $input_file = shift;
my $file_count = shift;
my %hash;
open(INPUT, "<$input_file") || die "Can't open file $input_file";
while(my $line = <INPUT>) {
my $num = ceil(rand($file_count));
$hash{$num} .= $line
}
foreach my $i (1..$file_count) {
open(OUTPUT, ">$i.txt") || die "Can't open file $i.txt";
print OUTPUT $hash{$i};
close OUTPUT;
}
close INPUT;