I have a Perl script which I am trying to understand. I am quoting first few lines of the script (abc.pl)
#!/usr/bin/perl
use strict;
use DateTime;
use File::Temp;
use Math::Complex;
use Getopt::Long;
use TauP::Time;
use Seed::Response;
use lib '/usr/lib/perl5';
use Seismogram::SAC;
my(#input,$output);
GetOptions('input=s{,}' => \#input, 'output=s' => \$output);
open my $out_fp,">$output";
foreach my $file (<#input>)
{
my $phase = 'S';
if ( $file =~ /BHZ/ ) { $phase = 'P';}
my $decData = File::Temp->new();
decimateData($file,$decData);
my($sac) = readSAC($decData);
my($resp) = readResponse($sac);
}
I have a few files which I have listed in "list".
Can anyone please let me understand how should I input "list" and output filenames to the perl script.
my(#input,$output);
This line declares variables used on next line (#input is an array of input files, $output is scalar for output filename)
GetOptions('input=s{,}' => \#input, 'output=s' => \$output);
GetOptions is a function from module Getopt::Long and this line parses commandline options and fills previously declared variables. You should read module documentation
Related
I have read some arguments at the command line:
#!/usr/bin/env perl
use 5.0360;
use warnings FATAL => 'all';
use autodie ':default';
use Devel::Confess 'color';
use Getopt::ArgParse;
my $parser = Getopt::ArgParse->new_parser(
help => 'This script finds and pretty-prints a line within a CSV file',
);
$parser-> add_args(
['-file', '-f', required => 1, type => 'Scalar'],
['-regex', '-r', required => 1, type => 'Scalar'], # regex pattern within each line
['-sep' , '-s', required => 0, type => 'Scalar']
);
my $args = $parser->parse_args(#ARGV);
my $sep = $args->sep // ',';
my $regex = $args->regex;
my #header;
say $args->regex;
open my $fh, '<', $args->file;
while (<$fh>) {
chomp;
if ($. == 1) {
#header = split $sep;
next
}
next unless /$regex/; # key point
say __LINE__;
# code later
}
the problem here is that the regex comparison
next unless /$regex/
works, but
next unless /$args->regex/
does not work, even though the two are copies of each other via my $regex = $args->regex;
Strangely, I can use $args->file but cannot use $args->regex
Why does $args->regex work like this? What is the correct terminology for $args->regex? It's not a hash, it's some sort of object.
You can't interpolate method calls into a string. I guarantee you that $args->file doesn't work either here. Note that $args->regex is not a field access but a method call, equivalent to $args->regex().
When interpolating a variable into a quote-like context (including regexes), you can use scalars, arrays, and array/hash field accesses.
I think that using a named variable my $regex = $args->regex is the clearest solution here. But if you really want to interpolate arbitrary expressions into a string, you can use a trick: you can interpolate arrays #foo, thus also an array dereference #{ ... }. Then:
next unless /#{[ $args->regex ]}/;
First off, my background is primarily in Python and I am relatively new at using Perl.
I am using tcsh for passing options into my .pl file.
In my code I have:
if( scalar #ARGV != 1)
{
help();
exit;
}
# Loading configuration file and parameters
our %configuration = load_configuration($ARGV[0]);
# all the scripts must be run for every configuration
my #allConfs = ("original");
sub Collimator
{
my $z = -25.0;
my %detector = init_det();
$detector{"pos"} = "0*cm 0.0*cm $z*cm"
}
foreach my $conf ( #allConfs )
{
$configuration{"variation"} = $conf ;
#other geometries ....
Collimator();
}
I want to add something that allows me to change the parameters of the subroutine in the .pl file from the command line. Currently, to generate the geometry I pass the following command into the tcsh CLI: perl file.pl config.dat. What I want is to be able to pass in something like this: perl file.pl config.dat -20.0.
I'm thinking that I need to add something to the effect of:
if($ARGV[1]){
$z = ARGV[1]}
However, I am not sure how to properly implement this. Is this something that I would specify within the subroutine itself or outside of it with the loading of the configuration file?
Use a library to handle command-line arguments, and Getopt::Long is excellent
use warnings;
use strict;
use Getopt::Long;
my ($config_file, #AllConfs);
my ($live, $dry_run) = (1, 0);
my $z;
GetOptions(
'config-file=s' => \$config_file,
'options-conf=s' => \#AllConfs,
'geom|g=f' => \$z,
'live!' => \$live,
'dry-run' => sub { $dry_run = 1; $live = 0 },
# etc
);
# Loading configuration file and parameters
# (Does it ===really=== need to be "our" variable?)
our %configuration = load_configuration($config_file);
my #data_files = #ARGV; # unnamed arguments, perhaps submitted as well
Options may be abbreviated as long as they are unambiguous so with my somewhat random sample above the program can be invoked for example as
prog.pl --conf filename -o original -o bare -g -25.0 data-file(s)
(Or whatever options for #AllConf are.) Providing explicitly g as another name for geom makes it a proper name, not an abbreviation (it doesn't "compete" with others).
Note that one can use -- or just -, and choose shorter or long(er) names for convenience or clarity etc. We get options, and there is a lot more than this little scoop, see docs.
Once the (named) options have been processed the rest on the command line is available in #ARGV, so one can mix and match arguments that way. Unnamed arguments are often used for file names. (The module offers a way to deal with those in some capacity as well.)
if ( #ARGV != 2 ) {
help();
}
my ( $path, $z ) = #ARGV;
perl gets input as list, so you can get its value using #ARGV or using $ARGV[0] and $ARGV[1]
die if ($#ARVG != 1);
#the first argument
print "$ARGV[0]\n";
#the second argument
print "$ARGV[1]\n"
Below is the Perl code in .pm file which is supposed to replace the specified strings (that are in "quotes") with some values. But its not happening. Can anyone explain what is happening in this code?
package SomePackage;
require Exporter;
#ISA = qw(Exporter);
#EXPORT = qw(send_request, create_mmd_and_transfer, update_mmd_file);
sub send_request {
my ( $service, $action, $torole ) = #_;
my ( $seller_request_mmd );
my $replace_contents = ();
$replace_contents{"REPLACE_Service"} = $service;
$replace_contents{"REPLACE_RequestAction"} = $action;
$replace_contents{"REPLACE_TradingPartner"} = $torole;
$replace_contents{"REPLACE_Requestxml"} = "Request.xml";
create_mmd_and_transfer( \%replace_contents, $seller_request_mmd, "/MMD.xml" );
}
sub create_mmd_and_transfer {
my $local_replace_contents = shift;
my $input_mmd = shift;
my $local_output_mmd = shift;
my $output_mmd = shift;
update_mmd_file( "$input_mmd", "temp_mmd_file.xml", $local_replace_contents );
}
sub update_mmd_file {
my $input_file = shift;
my $output_file = shift;
my $contents = shift;
open( MMD_FILE, "<$input_file" )
or main::error_exit(" Cannot open MMD file template $input_file \n $input_file not found int the Templates folder \n Please place the same and then run the script ");
open( TEMP_MMD_FILE, ">$output_file" );
while ( <MMD_FILE> ) {
s/^M//g; # Getrid of the ^Ms
foreach my $content ( keys( %$contents ) ) {
my $exact_value = ${%$contents}{$content};
if ( $main::test_scenario =~ /^Invalid Request Action \a\n\d Service/
and ( $content =~ /REPLACE_Service|REPLACE_RequestAction/i ) ) {
}
else {
if ( $exact_value ne "" ) {
s/$content/$exact_value/g;
}
}
}
print TEMP_MMD_FILE;
}
close MMD_FILE;
close TEMP_MMD_FILE;
}
The following will not make your script work, just create the better base for some future questions.
Before you even thinking about posting a perl question here:
1.) add to the top of your script:
use strict;
use warnings;
Posting a code here without these two lines, nobody will bother even trying to read the code.
2.) use perl -c SomePackage.pm for the check. If it will tell you: SomePackage.pm syntax OK - you can start thinking about posting a question here. ;)
Some basic problems with your script:
package SomePackage;
use strict; # see the above
use warnings;
require Exporter;
# these variables are defined outside of this package, so, tell perl this fact. use the `our`
our #ISA = qw(Exporter);
#the use warnings will warn you about the following line
# #EXPORT = qw(send_request, create_mmd_and_transfer, update_mmd_file);
#the correct one is without commas
our #EXPORT = qw(send_request create_mmd_and_transfer update_mmd_file); #not saying anything about the #EXPORT rudeness. :)
#my $replace_contents = ();
#the $replace_contents is a scalar. Bellow you using a hash. So,
my %replace_contents;
#or use the scalar but the lines bellow should use the hashref notation, e.g.
# $replace_contents->{"REPLACE_Service"} = $service;
# you decide. :)
# the seller_request_mmd contains undef here.
create_mmd_and_transfer( \%replace_contents, $seller_request_mmd, "/MMD.xml");
# also bellow, in the subroutine definition it wants 4 arguments.
# indicates a problem...
# using 2-arg open is not the best practice.
# Also, you should to use lexical filehandles
# open (MMD_FILE, "<$input_file")
# better
open (my $mmd_file, '<', $input_file)
# of course, you need change every MMD_FILE to $mmd_file
# check the result of the open and die if not successful
# or you can use the
use autodie;
# instead of $exact_value = ${%$contents}{$content};
# you probably want
my $exact_value = $contents->{$content};
Indent your code!
All the above are just about the syntactic problems and not solving anything about the "logic" of your code.
Ps: And me is still an beginner, so, others sure will find much more problems with the above code.
Ok. Here's what I've done to test this.
Firstly, you didn't give us an input file or the code that you use to call the module. So I invented them. I made the simplest possible input file:
REPLACE_Service
REPLACE_RequestAction
REPLACE_TradingPartner
REPLACE_Requestxml
And this driver program:
#!/usr/bin/perl
use strict;
use warnings;
use SomePackage;
send_request('foo', 'bar', 'baz');
sub error_exit {
die #_;
}
The first time, I ran it, I got this error:
Undefined subroutine &main::send_request called at test line 8.
That was because your #EXPORT line was wrong. You had:
#EXPORT = qw(send_request, create_mmd_and_transfer, update_mmd_file);
But the point of qw(...) is that you don't need the commas. So I corrected it to:
#EXPORT = qw(send_request create_mmd_and_transfer update_mmd_file);
Then I re-ran the program and got this error:
Cannot open MMD file template
not found int the Templates folder
Please place the same and then run the script at test line 11.
That looked like there was something missing. I changed the error message, adding indicators of where the variable interpolation was supposed to happen:
open( MMD_FILE, "<$input_file" )
or main::error_exit(" Cannot open MMD file template <$input_file> \n <$input_file> not found int the Templates folder \n Please place the same and then run the script ");
Then the error message looked like this:
Cannot open MMD file template <>
<> not found int the Templates folder
Please place the same and then run the script at test line 11.
So it seems clear that the $input_file variable isn't set in the update_mmd_file() subroutine. Tracing that variable back, we see that this value is originally the $seller_request_mmd variable in send_request(). But in send_request() you declare $seller_request_mmd but you never give it a value. So let's do that:
my ( $seller_request_mmd ) = 'test_input.txt';
Now, when I run your program, it runs to completion without any errors. And I find a new temp_mmd_file.xml is generated. But it is exactly the same as the input file. So more investigation is needed.
Digging into the update_mmd_file() subroutine, we find this interesting line:
my $exact_value = ${%$contents}{$content};
I think you're trying to extract a value from $contents, which is a hash reference. But your syntax is wrong. You were probably aiming at:
my $exact_value = ${$contents}{$content};
But most Perl programmers prefer the arrow notation for working with reference look-ups.
my $exact_value = $contents->{$content};
Making that change and re-running the program, I get an output file that contains:
foo
bar
baz
Request.xml
Which is exactly what I expected. So the program now works.
But there is still a lot of work to do. As you have been told repeatedly, you should always add:
use strict;
use warnings;
to your code. That will find a lot of potential problems in your code - which you should fix.
To be honest, this feels to me like you were trying to run before you could walk. I'd recommend spending some time to work through a good Perl introductory book before taking on my more Perl work.
And there was a lot of useful information missing from your question. It wouldn't have taken as long to get to the solution if you had shown us your driver program and your input data.
This is a pure Perl parameters passing issue. I cannot use Get::Opt as it is not installed on every machine.
I need to pass parameters with spaces and other special chars sometimes. Three scripts to demo the process. Is there a better way to do this?
[gliang#www stackoverflow]$ perl parameter_wrapper.pl
prep.pl #<5> parameters
prep_v2.pl #<5> parameters
<aaa_777-1>
<bbb-6666-2>
<Incomplete QA>
<-reason>
<too long, mail me at ben#example.com :)>
cat parameter_wrapper.pl
#!/usr/bin/perl -w
use strict;
# call prep.pl with 5 parameters
my $cmd = "./prep.pl aaa_777-1 bbb-6666-2 'Incomplete QA' -reason 'too long, mail me at ben\#example.com :)\n'";
system($cmd);
cat prep.pl
#!/usr/bin/perl -w
use strict;
my #parameters = #ARGV;
my $count = scalar(#parameters);
my #parameters_new = wrap_parameters(#parameters);
my $cmd = "./prep_v2.pl #parameters_new";
print "prep.pl #<$count> parameters\n";
system($cmd);
sub wrap_parameters {
my #parameters = #_;
my #parameters_new;
foreach my $var(#parameters) {
$var = quotemeta($var);
push(#parameters_new, $var);
}
return #parameters_new;
}
cat prep_v2.pl
#!/usr/bin/perl -w
use strict;
my #parameters = #ARGV;
my $count = scalar(#parameters);
print "prep_v2.pl #<$count> parameters\n";
foreach my $var (#parameters) {
#print "<$var>\n";
}
Getopt::Long has been part of the Perl core since Perl 5 was first released in 1994. Are you sure it's not available on the machines you're looking to deploy on? In your comment you refer to it as "Get::Opt", so could you have made a mistake while checking the machines?
This is an extension to my older question: Pass a hash object from one perl script to another using system. #Sobrique answered my question there.
Now, I want to know if I can pass an hash object to another perl script in the same way as my question above but using bsub.
wrapper script: script1.pl
use strict;
use warnings;
use Storable qw ( freeze );
use MIME::Base64;
my %test_hash = (
"fish" => "paste",
"apples" => "pears"
);
my $frozen = encode_base64 freeze( \%test_hash );
my ${string} = 'sample1';
# instead of just using system like this
# system("perl", "some_other_script.pl", $frozen);
# I want to use something like this:
system('bsub','-J',${string},'-o','${string}.out','-e','${string}.err','perl','some_other_script.pl',$frozen);
some_other_script.pl
use strict;
use warnings;
use Storable qw ( thaw );
use Data::Dumper;
use MIME::Base64;
# should read the imported hash correctly and print it out
my ($imported_scalar) = #ARGV;
print $imported_scalar;
my %param = %{ thaw (decode_base64 $imported_scalar ) };
print $param{'fish'};
print "\n";
How can I achieve this? Any help would be much appreciated.
Thanks!
This is really more a question about bsub than perl. Passing arguments via Base64 encoded parameters might do the trick, but actually if your purpose is passing data into the batch processing system, it might be worth using an input file:
If you have a look through the bsub manpage, you'll see:
-i input_file | -is input_file
That's probably a better way of handing 'input' to your batch job.