This is an extension to my older question: Pass a hash object from one perl script to another using system. #Sobrique answered my question there.
Now, I want to know if I can pass an hash object to another perl script in the same way as my question above but using bsub.
wrapper script: script1.pl
use strict;
use warnings;
use Storable qw ( freeze );
use MIME::Base64;
my %test_hash = (
"fish" => "paste",
"apples" => "pears"
);
my $frozen = encode_base64 freeze( \%test_hash );
my ${string} = 'sample1';
# instead of just using system like this
# system("perl", "some_other_script.pl", $frozen);
# I want to use something like this:
system('bsub','-J',${string},'-o','${string}.out','-e','${string}.err','perl','some_other_script.pl',$frozen);
some_other_script.pl
use strict;
use warnings;
use Storable qw ( thaw );
use Data::Dumper;
use MIME::Base64;
# should read the imported hash correctly and print it out
my ($imported_scalar) = #ARGV;
print $imported_scalar;
my %param = %{ thaw (decode_base64 $imported_scalar ) };
print $param{'fish'};
print "\n";
How can I achieve this? Any help would be much appreciated.
Thanks!
This is really more a question about bsub than perl. Passing arguments via Base64 encoded parameters might do the trick, but actually if your purpose is passing data into the batch processing system, it might be worth using an input file:
If you have a look through the bsub manpage, you'll see:
-i input_file | -is input_file
That's probably a better way of handing 'input' to your batch job.
Related
I want to take arguments as an array of hashes by using Getopt::Long in my script.
Consider the following command line example:
perl testing.pl --systems id=sys_1 ip_address=127.0.0.1 id=sys_2 ip_address=127.0.0.2
For the sake of simplicity, I'm using two systems and only two sub arguments of each system, i.e., id and ip_address. Ideally, the number of systems is dynamic; it may contain 1, 2 or more and so with the number of arguments of each system.
My script should handle these arguments in such a way that it will store in #systems array and each element will be a hash containing id and ip_address.
Is there any way in Getopt::Long to achieve this without parsing it myself?
Following is pseudocode for what I'm trying to achieve:
testing.pl
use Getopt::Long;
my #systems;
GetOptions('systems=s' => \#systems);
foreach (#systems) {
print $_->{id},' ', $_->{ip_address};
}
Here is an attempt, there might be more elegant solutions:
GetOptions('systems=s{1,}' => \my #temp );
my #systems;
while (#temp) {
my $value1 = shift #temp;
$value1 =~ s/^(\w+)=//; my $key1 = $1;
my $value2 = shift #temp;
$value2 =~ s/^(\w+)=//; my $key2 = $1;
push #systems, { $key1 => $value1, $key2 => $value2 };
}
for (#systems) {
print $_->{id},' ', $_->{ip_address}, "\n";
}
Output:
sys_1 127.0.0.1
sys_2 127.0.0.2
I actually think this is a design problem, more than a problem with GetOpt - the notion of supporting multiple, paired arguments passed as command line arguments I think is something that you'd be far better off avoiding.
There's a reason that GetOpt doesn't really support it - it's not a scalable solution really.
How about instead just reading the values from STDIN?:
#!/usr/bin/env perl
use strict;
use warnings;
use Data::Dumper;
my %systems = do { local $/; <DATA> } =~ m/id=(\w+) ip_address=([\d\.]+)/mg;
print Dumper \%systems;
And then you'd be able to invoke your script as:
perl testing.pl <filename_with_args>
Or similar.
And if you really must:
my %systems = "#ARGV" =~ m/id=(\w+) ip_address=([\d\.]+)/g;
Both of the above work for multiple parameters.
However, your comment on another post:
I can't because I'm fetching parameters from database and converting them into command line and then passing it to the script using system command $cmd_lines_args = '--system --id sys_1 --ip_address 127.0.0.0.1'; system("perl testing.pl $cmd_lines_args"); $cmd_lines_args I'll generate dynamically using for loop by reading from database
.. that makes this an XY Problem.
Don't do it like that:
open ( my $script, '|-', "perl testing.pl" );
print {$script} "id=some_id ip_address=10.9.8.7\n";
print {$script} "id=sys2 ip_address=10.9.8.7\n";
etc.
What you are describing,
--systems id=sys_1 ip_address=127.0.0.1 id=sys_2 ip_address=127.0.0.2
appears to be one option that takes a variable number of arguments that are pairs, and come in multiples of two. Getopt::Long's "Options with multiple values" lets you do the following:
GetOptions('systems=s{2,4}' => \#systems);
This lets you specify 2, 3 or 4 arguments, but it does not have syntax for "any even number of arguments" (to cover an arbitrary number of pairs beyond two), and you still have to unpack the id=sys_1 manually then. You can write a user-defined subroutine that handles the processing of --systems' arguments (but does not take into account missing id=...s):
my $system;
my %systems;
GetOptions('systems=s{,}' => sub {
my $option = shift;
my $pair = shift;
my ($key, $value) = split /=/, $pair;
$system = $value if $key eq 'id';
$systems{$system} = $value if $key eq 'ip_address';
});
I would however prefer one of the following schemes:
--system sys_1 127.0.0.1 --system sys_2 127.0.0.1
--system sys_1=127.0.0.1 --system sys_2=127.0.0.1
They're achieved with the following:
GetOptions('system=s{2}', \#systems);
GetOptions('system=s%', \#systems);
I would just parse the --systems arg and quote the "hashes" like this:
perl testing.pl --systems "id=s1 ip_address=127.0.0.1 id=s2 ip_address=127.0.0.2"
Parse like perhaps so:
my($systems,#systems);
GetOptions('systems=s' => \$systems);
for(split/\s+/,$systems){
my($key,$val)=split/=/,$_,2;
push #systems, {} if !#systems or exists$systems[-1]{$key};
$systems[-1]{$key}=$val;
}
print "$_->{id} $_->{ip_address}\n" for #systems;
Prints:
sys_1 127.0.0.1
sys_2 127.0.0.2
I have a Perl script which I am trying to understand. I am quoting first few lines of the script (abc.pl)
#!/usr/bin/perl
use strict;
use DateTime;
use File::Temp;
use Math::Complex;
use Getopt::Long;
use TauP::Time;
use Seed::Response;
use lib '/usr/lib/perl5';
use Seismogram::SAC;
my(#input,$output);
GetOptions('input=s{,}' => \#input, 'output=s' => \$output);
open my $out_fp,">$output";
foreach my $file (<#input>)
{
my $phase = 'S';
if ( $file =~ /BHZ/ ) { $phase = 'P';}
my $decData = File::Temp->new();
decimateData($file,$decData);
my($sac) = readSAC($decData);
my($resp) = readResponse($sac);
}
I have a few files which I have listed in "list".
Can anyone please let me understand how should I input "list" and output filenames to the perl script.
my(#input,$output);
This line declares variables used on next line (#input is an array of input files, $output is scalar for output filename)
GetOptions('input=s{,}' => \#input, 'output=s' => \$output);
GetOptions is a function from module Getopt::Long and this line parses commandline options and fills previously declared variables. You should read module documentation
I'd like to store file handle to a global hash and read() it in a subroutine without revealing CGI object, but I find that it doesn't work(resulting zero sized output file).
Here is the simplified perl code:
#!/usr/local/bin/perl
use CGI;
use vars qw(%in);
&init_hash;
$fname = &process_uploaded_file if($in{'upfile'});
$fsize = -s $fname;
print "Content-Type: text/plain\r\n\r\n";
print "in{'upfile'}=",$in{'upfile'},"\r\n";
print "in{'desc'}=",$in{'desc'},"\r\n";
print "fname=",$fname,"\r\n";
print "fsize=",$fsize,"\r\n";
sub init_hash{
my $q = new CGI;
$in{'desc'} = $q->param('desc');
$in{'upfile'} = $q->param('upfile');
$in{'upfh'} = $q->upload('upfile') if($in{'upfile'});
}
sub process_uploaded_file{
my $fname = time.'.bin';
open(OUT,'>',$fname) || die('open file failed');
while(my $read = read($in{'upfh'}, my $buff, 4096)){
print OUT $buff;
}
close(OUT);
eval { close($in{'upfh'}); };
return $fname;
}
EDIT: I should provide perl and cgi.pm version.
Perl version: This is perl 5, version 12, subversion 2 (v5.12.2) built for MSWin32-x86-multi-thread
(with 8 registered patches, see perl -V for more detail)
$CGI::VERSION='3.50';
There is so much wrong with your code.
First your problem: you are trying to optimize where optimization isn't due. And the temp files of the CGI object are deleted before you actually access them. Your code should work when you extend the lifetime of the CGI object, e.g. by adding it to the %in hash.
Always use strict; use warnings;. There are no excuses.
Global variables are declared with our. The vars pragma is a historical artifact. But please don't use global variables, as they are unneccessary here.
Don't call functions like &foo unless you can tell me what exactly this does. Until you have this knowledge: foo().
Use the header method of the CGI object to write headers: $q->header('text/plain').
The \n may not be what you think it is. Do a binmode STDOUT to remove the :crlf PerlIO-layer if it is currently applied. Although equivalent to \r\n, It may be clearer to write \015\012 to demonstrate that you care about the actual bytes.
You can interpolate variables into strings, you know. You can also specify a string that is to be appended after each print by setting $\:
{
local $\ = "\015\012";
print "in{'upfile'}=$in{'upfile'}";
print "in{'desc'}=$in{'desc'}";
print "fname=$fname";
print "fsize=$fsize";
}
Don't use bareword filehandles. Instead of open OUT, "<", $fname you should open my $outfh, "<", $fname.
Why did you put one close in an eval? I don't see how this should die.
Is there a (best) way to check, if $uri was passed in single quotes?
#!/usr/local/bin/perl
use warnings;
use 5.012;
my $uri = shift;
# uri_check
# ...
Added this example, to make my problem more clear.
#!/usr/local/bin/perl
use warnings;
use 5.012;
use URI;
use URI::Escape;
use WWW::YouTube::Info::Simple;
use Term::Clui;
my $uri = shift;
# uri check here
$uri = URI->new( $uri );
my %params = $uri->query_form;
die "Malformed URL or missing parameter" if $params{v} eq '';
my $video_id = uri_escape( $params{v} );
my $yt = WWW::YouTube::Info::Simple->new( $video_id );
my $info = $yt->get_info();
my $res = $yt->get_resolution();
my #resolution;
for my $fmt ( sort { $a <=> $b } keys %$res ) {
push #resolution, sprintf "%d : %s", $fmt, $res->{$fmt};
}
# with an uri-argument which is not passed in single quotes
# the script doesn't get this far
my $fmt = choose( 'Resolution', #resolution );
$fmt = ( split /\s:\s/, $fmt )[0];
say $fmt;
You can't; bash parses the quotes before the string is passed to the Perl interpreter.
To expand on Blagovest's answer...
perl program http://example.com/foo?bar=23&thing=42 is interpreted by the shell as:
Execute perl and pass it the arguments program and http://example.com/foo?bar=23
Make it run in the background (that's what & means)
Interpret thing=42 as setting the environment variable thing to be 42
You should have seen an error like -bash: thing: command not found but in this case bash interpreted thing=42 as a valid instruction.
The shell handles the quoting and Perl has no knowledge of that. Perl can't issue an error message, it just sees arguments after shell processing. It never even sees the &. This is just one of those Unix things you'll have to learn to live with. The shell is a complete programming environment, for better or worse.
There are other shells which dumb things down quite a bit so you can avoid this issue, but really you're better off learning the quirks and powers of a real shell.
I have a hosts file that looks like this:
10.10.10.1 myserver1 myserver1alias
10.10.10.2 myserver2 myserver2alias
I'm looking for a way using perl to pass in an argument of myserver1 and have it return myserver1alias, likewise if I pass in myserver2 it should return myserver2alias. Any suggestions?
From the command line:
$ perl -lane 'print $F[-1] if $F[1] eq "myserver1"' /etc/hosts
while(<>){
if (/myserver1/){
#s = split /myserver1/,$_,2;
print $s[-1];
}
}
The quick-and-dirty way is:
perl -nE 'say $1 if /myserver1\s+(\w+)$/' path/to/hostfile
You might need to do queries similar to this from time to time, so you could probably make a reusable chunk of code to do this for you. Something like:
#!/usr/bin/perl
use strict;
use warnings;
use 5.10;
use HostFileParser;
my $host = HostFileParser->parse("path/to/hostfile")
my $server = $host->find(server => "myserver1")
say $server->alias;
Of course, I'm not going to "give you teh codez" for all of that. ;)
(These answers assume Perl 5.10, but if you don't have it the changes are just say $x => print "$x\n", or sub say { print "#_\n" }.)