Passing hash reference as an argument to perl script from perl script - perl

I want to pass a hash reference as an argument from one perl script (script1.pl) to another perl script (script2.pl). This is how my code looks:
----------------------------script1.pl---------------------------------
#!/usr/bin/perl -w
use strict;
use warnings;
my %hash = (
'a' => "Harsha",
'b' => "Manager"
);
my $ref = \%hash;
system "perl script2.pl $ref";
----------------------------script2.pl---------------------------------
#!/usr/bin/perl -w
use strict;
use warnings;
my %hash = %{$ARGV[0]};
my $string = "a";
if (exists($hash{$string})){
print "$string = $hash{$string}\n";
}
And this is the output error:
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `perl script2.pl HASH(0x8fbed0)'
I can't figure out the right way to pass the reference.

A hash is an in memory data structure. Processes 'own' their own memory space, and other processes can't just access it. If you think about it, I'm sure you'll spot why quite quickly.
A hash reference is an address of that memory location. Even if the other process could 'understand' it, it still wouldn't be able to access the memory space.
What we're talking about here is actually quite a big concept - Inter Process Communication or IPC - so much so there's a whole chapter of the documentation about it, called perlipc.
The long and short of it is this - you can't do what you're trying to do. Sharing memory between processes is much more difficult than you imagine.
What you can do is transfer the data back and forth - not by reference, but the actual information contained.
I would suggest that for your example, the tool for the job is JSON, because then you can encode and decode your hash:
#!/usr/bin/perl -w
use strict;
use warnings;
use JSON;
my %hash = (
'a' => "Harsha",
'b' => "Manager"
);
my $json_string = to_json( \%hash );
print $json_string;
This gives:
{"b":"Manager","a":"Harsha"}
Then your can 'pass' your $json_string - either on the command line, although bear in mind that any spaces in it confuses #ARGV a bit if you're not careful - or via STDIN.
And then decode in your sub process:
use strict;
use warnings;
use JSON;
my $json_string = '{"b":"Manager","a":"Harsha"}';
my $json = from_json ( $json_string );
my $string = "a";
if (exists($json -> {$string} )){
print "$string = ",$json -> {$string},"\n";
}
(You can make it more similar to your code by doing:
my $json = from_json ( $json_string );
my %hash = %$json;
Other options would be:
use Storable - either freezing and thawing ( memory) or storing and retrieving (disk)
use IPC::Open2 and send data on STDIN.
There's a variety of options really - have a look at perlipc. But it's not as simple a matter as 'just passing a reference' unfortunately.

Use Storable to store data in first script and retrieve it from other.
firstscript.pl
store (\%hash, "/home/chankey/secondscript.$$") or die "could not store";
system("perl", "secondscript.pl", $$) == 0 or die "error";
secondscript.pl
my $parentpid = shift;
my $ref = retrieve("/home/chankey/secondscript.$parentpid") or die "couldn't retrieve";
print Dumper $ref;
You've received the %hash in $ref. Now use it the way you want.

You can't pass a reference from one script to another - that reference only has meaning within the currently running instance of perl.
You would need to "serialise" the data in the first script, and then "deserialise" it in the second.

Your way of calling perl file is wrong.
Just change the way of calling it and you are done.
Script1.pl
---------------------------------
#!/usr/bin/perl -w
use strict;
use warnings;
my %hash = (
'a' => "Harsha",
'b' => "Manager"
);
system("perl","script2.pl",%hash);
Use this %hash in another perl script as shown below.
Script2.pl
----------------------------------
#!/usr/bin/perl -w
use strict;
use warnings;
my %hash = #ARGV;
my $string = "a";
if (exists($hash{$string})){
print "$string = $hash{$string}\n";
}
OutPut is
a = Harsha

Related

Using Storable to store hash and use it in another script

I am storing a hash in a script like so (part of code left out):
use strict;
use warnings;
use utf8;
use Storable qw(nstore_fd);
open( my $fh, ">", 'hash.hash');
my $hash_ref = \%hash;
nstore_fd($hash_ref, $fh);
close $fh;
Which creates a file hash.hash. In another file, then, I try to 'import' the hash and read its values to be used (e.g. looping the hash, reading values, checking existence ...).
I tried something simple, i.e. read the hash, dereference and read its keys, and print those out. However I get the error Not a HASH reference at CreateSql.pl line 12. It is a scalar. But I don't see why. The documentation of the module states that a reference is returned.
To retrieve data stored to disk, use retrieve with a file name. The
objects stored into that file are recreated into memory for you, and a
reference to the root object is returned.
So what did I do wrong, and how do I get access to the stored hash?
use strict;
use warnings;
use utf8;
use Data::Dumper;
use Storable qw(retrieve);
my ($hash_path) = #ARGV;
my $hash_ref = retrieve($hash_path);
my #keys = keys % { $hash_ref }; # throws error
print Dumper(\#keys);
You really must show the code that has given you the problems that you describe, otherwise it becomes very hard to help you. The code in your question doesn't compile. If you want to make changes to the program before you publish it, the test it again to make sure that it at least compiles, and also display the problems you're reporting
The main problem is that you are writing binary data to a file opened as text. If you change open( my $fh, ">", 'hash.hash') to open( my $fh, ">:raw", 'hash.hash') then everything should work
This variant of your own code correctly stores and retrieves some sample data
use strict;
use warnings 'all';
use Storable qw/ nstore_fd retrieve /;
use constant STORAGE => 'hash.hash';
my %hash = ( a => 'b', c => 'd' );
{
open my $fh, '>:raw', STORAGE or die $!;
nstore_fd \%hash, $fh;
}
my $hash_ref = retrieve STORAGE;
use Data::Dump;
dd $hash_ref;
my #keys = keys % { $hash_ref }; # throws error
dd \#keys;
output
{ a => "b", c => "d" }
["a", "c"]
However, you've made some strange choices. There's no need to use an nstore function to store the data in "network order", and you could have avoided the problem altogether if you had let the module open the file itself
Here's a variant that just uses store and retrieve. The output is identical to that of the code above
use strict;
use warnings 'all';
use Storable qw/ store retrieve /;
use constant STORAGE => 'hash.hash';
my %hash = ( a => 'b', c => 'd' );
store \%hash, STORAGE;
my $hash_ref = retrieve STORAGE;
use Data::Dump;
dd $hash_ref;
my #keys = keys % { $hash_ref }; # throws error
dd \#keys;

Pass a hash object from one perl script to another using system

I have the following perl script, that takes in a parameters' file and stores it into a hash. I want to modify & pass this hash to another perl script that I am calling using the system command:
script1.pl
#!/usr/bin/perl -w
# usage perl script1.pl script1.params
# script1.params file looks like this:
# PROJECTNAME=>project_dir
# FASTALIST=>samples_fastq.csv
use Data::Dumper;
my $paramfile = $ARGV[0];
# open parameter file
open PARAM, $paramfile or die print $!;
# save it in a hash
my %param;
while(<PARAM>)
{
chomp;
#r = split('=>');
$param{$r[0]}=$r[1];
}
# define directories
# add to parameters' hash
$param{'INDIR'} = $param{'PROJECTNAME'}.'/input';
$param{'OUTDIR'} = $param{'PROJECTNAME'}.'/output';
.... do something ...
# #samples is a list of sample names
foreach (#samples)
{
# for each sample, pass the hash values & sample name to a separate script
system('perl script2.pl <hash> $_');
}
script2.pl
#!/usr/bin/perl -w
use Data::Dumper;
## usage <script2.pl> <hash> <samplename>
# something like getting and printing the hash
my #string = $ARGV[0];
print #string;
If you can help me showing how to pass and get the hash object (something simple like printing the hash object in the second script would do), then I'd appreciate your help.
Thanks!
What you're looking for is something called serialisation. It's difficult to directly represent a memory structure in such a way as to pass it between processes, because of all sorts of fun things like pointers and buffers.
So you need to turn your hash into something simple enough to hand over in a single go.
Three key options for this in my opinion:
Storable - a perl core module that lets you freeze and thaw a data structure for this sort of purpose.
JSON - a text based representation of a hash-like structure.
XML - bit like JSON, but with slightly different strengths/weaknesses.
Which you should use depends a little on how big your data structure is.
Storable is probably the simplest, but it's not going to be particularly portable.
There's also Data::Dumper that's an option too, as it prints data structures. Generally though, I'd suggest that has all the downsides of all the above - you still need to parse it like JSON/XML but it's also not portable.
Example using Storable:
use strict;
use warnings;
use Storable qw ( freeze );
use MIME::Base64;
my %test_hash = (
"fish" => "paste",
"apples" => "pears"
);
my $frozen = encode_base64 freeze( \%test_hash );
system( "perl", "some_other_script.pl", $frozen );
Calling:
use strict;
use warnings;
use Storable qw ( thaw );
use Data::Dumper;
use MIME::Base64;
my ($imported_scalar) = #ARGV;
print $imported_scalar;
my $thing = thaw (decode_base64 $imported_scalar ) ;
print Dumper $thing;
Or:
my %param = %{ thaw (decode_base64 $imported_scalar ) };
print Dumper \%param;
This will print:
BAoIMTIzNDU2NzgEBAQIAwIAAAAKBXBhc3RlBAAAAGZpc2gKBXBlYXJzBgAAAGFwcGxlcw==
$VAR1 = {
'apples' => 'pears',
'fish' => 'paste'
};
Doing the same with JSON - which has the advantage of being passed as plain text, and in a general purpose format. (Most languages can parse JSON):
#!/usr/bin/env perl
use strict;
use warnings;
use JSON;
my %test_hash = (
"fish" => "paste",
"apples" => "pears"
);
my $json_text = encode_json ( \%test_hash );
print "Encoded: ",$json_text,"\n";
system( "perl", "some_other_script.pl", quotemeta $json_text );
Calling:
#!/usr/bin/env perl
use strict;
use warnings;
use JSON;
use Data::Dumper;
my ($imported_scalar) = #ARGV;
$imported_scalar =~ s,\\,,g;
print "Got: ",$imported_scalar,"\n";
my $thing = decode_json $imported_scalar ;
print Dumper $thing;
Need the quotemeta and the removal of slashes unfortunately, because the shell interpolates them. This is the common problem if you're trying to do this sort of thing.

Why does Test::MockObject make thawing my objects throw warnings?

This one needs a bit of explanation to start with. I've got a unit test where I save Class::Std::Fast::Storable objects that come from SOAP::WSDL using Storable. The object I am storing is the result of a webservice call. It ends up being encoded with MIME::Base64 and written somewhere to a file. This is working great.
When I was building up the unit test, I needed to use Test::MockObject to mock the call that webservice, thus returning the restored object. But somehow this is throwing a bunch of warnings about the use of uninitialized value in hash element.
I tried recreating it as a small example. This first bit of code is how I get the base64 output for the example. We will use it in a minute.
use strict;
use warnings;
use MIME::Base64;
use Storable;
use SOAP::WSDL::XSD::Typelib::Builtin::anySimpleType;
my $object = SOAP::WSDL::XSD::Typelib::Builtin::anySimpleType->new;
$object->set_value('foo');
print encode_base64(Storable::freeze($object));
So we got three lines of base64. Let's try to restore them:
use strict;
use warnings;
use MIME::Base64;
use Storable;
use Test::Simple tests => 1;
local $/ = undef;
my $object = Storable::thaw(decode_base64(<DATA>));
ok( $object->get_value, 'foo' );
__DATA__
BAgIMTIzNDU2NzgECAgIE0ADAQAAAAQDAQAAAAoDZm9vBQAAAHZhbHVlMAAAAFNPQVA6OldTREw6
OlhTRDo6VHlwZWxpYjo6QnVpbHRpbjo6YW55U2ltcGxlVHlwZYAwU09BUDo6V1NETDo6WFNEOjpU
eXBlbGliOjpCdWlsdGluOjphbnlTaW1wbGVUeXBlEAQICDEyMzQ1Njc4BAgICAUBAAAAAQ==
Neat. It works!
~> perl foo.t
1..1
ok 1 - foo
Now let's add Test::MockObject.
use strict;
use warnings;
use MIME::Base64;
use Storable;
use Test::Simple tests => 1;
use Test::MockObject; # <------- only line I changed
local $/ = undef;
my $object = Storable::thaw(decode_base64(<DATA>));
ok( $object->get_value, 'foo' );
__DATA__
BAgIMTIzNDU2NzgECAgIE0ADAQAAAAQDAQAAAAoDZm9vBQAAAHZhbHVlMAAAAFNPQVA6OldTREw6
OlhTRDo6VHlwZWxpYjo6QnVpbHRpbjo6YW55U2ltcGxlVHlwZYAwU09BUDo6V1NETDo6WFNEOjpU
eXBlbGliOjpCdWlsdGluOjphbnlTaW1wbGVUeXBlEAQICDEyMzQ1Njc4BAgICAUBAAAAAQ==
Ok, this is weird. It works, but it throws an error.
1..1
Use of uninitialized value in hash element at /usr/lib/perl5/site_perl/5.16.2/SOAP/WSDL/XSD/Typelib/Builtin/anySimpleType.pm line 53, <DATA> chunk 1.
ok 1 - foo
So I looked at line 53 of anySimpleType.pm, and it says:
my $OBJECT_CACHE_REF = Class::Std::Fast::OBJECT_CACHE_REF();
sub new {
my $self = pop #{ $OBJECT_CACHE_REF->{ $_[0] } }; # <-- here
$self = bless \(my $o = Class::Std::Fast::ID()), $_[0]
if not defined $self;
Hmm. $_[0] is undef. Looks like new was called without an argument.
But how the hell can loading Test::MockObject do that? Or maybe that warning is always popping up, but somehow it was not shown before? I debugged it a little, and it turns out the warning is always showing in Komodo IDEs debugger, regardless of what I loaded.
However, it only shows up in the normal program output if I have Test::MockObject loaded as well. Can anyone explain that to me?
I still don't know why this is happening exactly. My debugging led me to believe that the initialization warnings are always thrown by these Storable objects. However, they are silent if Test::MockObject is not there.
So the workaround to get it to shut up is as follows:
local $SIG{__WARN__} = sub { warn $_[0] unless $_[0] =~ /uninitialized/};
local $/ = undef;
my $object = Storable::thaw(decode_base64(<DATA>));
ok( $object->get_value, 'foo' );

Getting variable uninitialized in Perl using stat

I'm working on a script which shows the age of an existing file and then automatically delete it after about 4 hours of being there (presumed maintenance window). I've been trying to test my output with the stat function in Perl. We have multiple boxes, some running Linux and Solaris, so this is the most portable way to do it. I'm trying to get epoch time.
use File::stat;
$file = "samplefile";
$mtime = (stat($file))[9];
if (!defined $file){
print "No file specified\n";
exit 1;
}
printf("%d", #mtime);
I know stat() returns a #_, so I tried to change my datatype to this. But it keeps coming back saying that mtime has not been initialized. Why does it say that?
You print the contents of #mtime, but you placed the result in $mtime. Always use use strict; use warnings;. It would have found your problem.
Your code should look like one of the following:
Without File::stat (stat returns a list):
use strict;
use warnings;
my $file = "samplefile";
my $mtime = (stat($file))[9];
die "Can't stat file: $!\n" if !defined($mtime);
printf("%d\n", $mtime);
(EXPR)[9] returns the 10th element of the list EXPR returns, or undef if the list isn't that long. This is a scalar you assigned to $mtime, not #mtime.
With File::stat (stat returns an object):
use strict;
use warnings;
use File::stat;
my $file = "samplefile";
my $stat = stat($file);
die "Can't stat file: $!\n" if !$stat;
printf("%d\n", $stat->mtime);
You are accessing the array #mtime, but you assign a value to the scalar $mtime. They are not the same variable.
If you had used
use strict;
You would be aware of this problem right away:
Global symbol "#mtime" requires explicit package name at ...
You should always use
use strict;
use warnings;
Not doing so will cause you lots of extra work and "mysterious" bugs.

Write an anonymous sub in Perl to a file for later use

I have a Perl program that generates parsing rules as subs from an input file. The subs are anonymously defined an put into a hash. Now, I want to export that hash, with all the subs and then load them again later to use with a different program.
How do I go about doing this? Is there some way to extract the code of each sub, or can I copy the block of memory that the hash exists in and then cast it as a hash when I load it again later?
Thanks in advance.
KiokuDB can handle storing code references in addition to other Perl types. So can the various YAML modules, Data::Dump::Streamer, and even Data::Dumper.
From the "Code References" section of the Storable documentation (with added emphasis):
Since Storable version 2.05, CODE references may be serialized with the help of B::Deparse. To enable this feature, set $Storable::Deparse to a true value. To enable deserialization, $Storable::Eval should be set to a true value. Be aware that deserialization is done through eval, which is dangerous if the Storable file contains malicious data.
In the demo below, a child process creates the hash of anonymous subs. Then the parent—in an entirely separate process and address space, so it can't see %dispatch—reads the output from freeze in the same way that you might from a file on disk.
#! /usr/bin/perl
use warnings;
use strict;
use Storable qw/ freeze thaw /;
my $pid = open my $fh, "-|";
die "$0: fork: $!" unless defined $pid;
if ($pid == 0) {
# child process
my %dispatch = (
foo => sub { print "Yo!\n" },
bar => sub { print "Hi!\n" },
baz => sub { print "Holla!\n" },
);
local $Storable::Deparse = 1 || $Storable::Deparse;
binmode STDOUT, ":bytes";
print freeze \%dispatch;
exit 0;
}
else {
# parent process
local $/;
binmode $fh, ":bytes";
my $frozen = <$fh>;
local $Storable::Eval = 1 || $Storable::Eval;
my $d = thaw $frozen;
$d->{$_}() for keys %$d;
}
Output:
Hi!
Holla!
Yo!