How can I store simple hash in perl using data dumper - perl

%hash = ('abc' => 123, 'def' => [4,5,6]);
how can I store above hash in file using data dumper in Perl

Files can only contain sequences of bytes, so you need to convert the data structure into a sequence of bytes somehow. This process is called serialization.
The possibilities available to you are endless, but a few are worth mentioning:
JSON is a very common choice.
YAML is more flexible.
Storable is specifically made for Perl data structures.
There is also Data::Dumper, as you say.
use Data::Dumper qw( );
sub serialize {
my ($x) = #_;
local $Data::Dumper::Purity = 1; # Required for some data structures.
local $Data::Dumper::Useqq = 1; # Optional. Limits output to ASCII.
local $Data::Dumper::Sortkeys = 1; # Optional. Makes revision control easier.
return Data::Dumper->Dump([$x], ["x"]);
}
print($fh serialize($x));
Data::Dumper isn't a particularly good choice, since there's no existing module to safely deserialize the structure in Perl[1], and there's even less support outside of Perl.
sub deserialize {
my ($s) = #_;
my $x;
eval($s); # XXX Unsafe!
die($#) if $#;
return $x;
}
If you're ok with limiting yourself to data structure JSON can serialize (by setting Purity to 0), then you could use Data::Undump to safely deserialize. But then why not just use JSON?!

use Data::Dumper
open (FL, ">", "file.txt") or die "Cant open file $! ";
print FL Dumper \%hash;
close FL;

"how can I store above hash in file using data dumper in Perl"
Store it as JSON so it can be read back by (almost) anything, using Data::Dumper configured to print JSON.
use strict;
use warnings;
use Data::Dumper;
local $Data::Dumper::Pair = ' : ';
local $Data::Dumper::Quotekeys = 1;
local $Data::Dumper::Useqq = 1;
local $Data::Dumper::Terse = 1;
my %hash = ('abc' => 123, 'def' => [4,5,6]);
open my $file, '>', 'foo.json' or die $!;
print $file Dumper \%hash;
Output:
$ cat foo.json
{
"def" : [
4,
5,
6
],
"abc" : 123
}
(Note: I would of course rather use a dedicated JSON-handling module for this, but you asked ....)

Related

Using Storable to store hash and use it in another script

I am storing a hash in a script like so (part of code left out):
use strict;
use warnings;
use utf8;
use Storable qw(nstore_fd);
open( my $fh, ">", 'hash.hash');
my $hash_ref = \%hash;
nstore_fd($hash_ref, $fh);
close $fh;
Which creates a file hash.hash. In another file, then, I try to 'import' the hash and read its values to be used (e.g. looping the hash, reading values, checking existence ...).
I tried something simple, i.e. read the hash, dereference and read its keys, and print those out. However I get the error Not a HASH reference at CreateSql.pl line 12. It is a scalar. But I don't see why. The documentation of the module states that a reference is returned.
To retrieve data stored to disk, use retrieve with a file name. The
objects stored into that file are recreated into memory for you, and a
reference to the root object is returned.
So what did I do wrong, and how do I get access to the stored hash?
use strict;
use warnings;
use utf8;
use Data::Dumper;
use Storable qw(retrieve);
my ($hash_path) = #ARGV;
my $hash_ref = retrieve($hash_path);
my #keys = keys % { $hash_ref }; # throws error
print Dumper(\#keys);
You really must show the code that has given you the problems that you describe, otherwise it becomes very hard to help you. The code in your question doesn't compile. If you want to make changes to the program before you publish it, the test it again to make sure that it at least compiles, and also display the problems you're reporting
The main problem is that you are writing binary data to a file opened as text. If you change open( my $fh, ">", 'hash.hash') to open( my $fh, ">:raw", 'hash.hash') then everything should work
This variant of your own code correctly stores and retrieves some sample data
use strict;
use warnings 'all';
use Storable qw/ nstore_fd retrieve /;
use constant STORAGE => 'hash.hash';
my %hash = ( a => 'b', c => 'd' );
{
open my $fh, '>:raw', STORAGE or die $!;
nstore_fd \%hash, $fh;
}
my $hash_ref = retrieve STORAGE;
use Data::Dump;
dd $hash_ref;
my #keys = keys % { $hash_ref }; # throws error
dd \#keys;
output
{ a => "b", c => "d" }
["a", "c"]
However, you've made some strange choices. There's no need to use an nstore function to store the data in "network order", and you could have avoided the problem altogether if you had let the module open the file itself
Here's a variant that just uses store and retrieve. The output is identical to that of the code above
use strict;
use warnings 'all';
use Storable qw/ store retrieve /;
use constant STORAGE => 'hash.hash';
my %hash = ( a => 'b', c => 'd' );
store \%hash, STORAGE;
my $hash_ref = retrieve STORAGE;
use Data::Dump;
dd $hash_ref;
my #keys = keys % { $hash_ref }; # throws error
dd \#keys;

perl storing hash in %ENV

I want to (ab-)use the global %ENV to store a hash. This seems to work differently for %ENV than for ordinary hashes. in the program below, the $ENV{myhash} still contains 'myhash' => 'HASH(0x7ffad8807380)' and the %ahash is still around. is it possible to convert the hex address back to point at its location, instead of just containing the string? I guess I could serialize and unserialize the hash instead. what is the right way to do this?
#!/usr/bin/perl -w
use common::sense;
use Data::Dumper;
my %QENV = ( nohash => 'noh' );
my %ahash= ( hv1 => 'htext1', hv2 => 'htext2' );
$QENV{'myhash'} = \%ahash;
print "works: ". Dumper(\%QENV)."\n\n\n";
$ENV{'myhash'} = \%ahash;
print "fails: ". Dumper(\%ENV)."\n";
%ENV is a magical hash. It reflects the process's environment. Reading from it reads from the environment, and writing to it changes the environment.
If you can guarantee the referenced variable is still alive (by it still being in scope or by it having its REFCNT increased), you can indeed create a reference to it from the address.
use strict;
use warnings;
use Data::Dumper;
use Inline C => <<'__EOS__';
SV* _newRV(IV iv) {
return newRV((SV*)iv);
}
__EOS__
my %hash = ( a => 1, b => 2 );
my $ref = \%hash;
my $ref_string = "$ref";
my $addr = hex( ( $ref_string =~ /\((.*)\)/ )[0] );
my $ref2 = _newRV($addr);
print(Dumper($ref2));
I have no idea why you'd want to do this. It would not permit another process to access the data since one process can't access the memory of another.
You seem to want to share data. Here's an example I put out there often that shows how to store data in a JSON file, then retrieve it. JSON is cross-language, so the data can be used by many programming languages, not just Perl. Although this example is within a single script, imagine it being two different Perl applications:
use warnings;
use strict;
use JSON;
my $file = 'data.json';
my %h = (
foo => 'bar',
raz => {
ABC => 123,
XYZ => [4, 5, 6],
}
);
my $json = encode_json \%h;
# write the JSON to a file, and close it
open my $fh, '>', $file or die $!;
print $fh $json;
close $fh or die $!;
# open the JSON file, and read it
open my $json_file, '<', $file or die $!;
my $read_json;
{
local $/;
$read_json = <$json_file>;
}
my $perl_hash = decode_json $read_json;

Pass a hash object from one perl script to another using system

I have the following perl script, that takes in a parameters' file and stores it into a hash. I want to modify & pass this hash to another perl script that I am calling using the system command:
script1.pl
#!/usr/bin/perl -w
# usage perl script1.pl script1.params
# script1.params file looks like this:
# PROJECTNAME=>project_dir
# FASTALIST=>samples_fastq.csv
use Data::Dumper;
my $paramfile = $ARGV[0];
# open parameter file
open PARAM, $paramfile or die print $!;
# save it in a hash
my %param;
while(<PARAM>)
{
chomp;
#r = split('=>');
$param{$r[0]}=$r[1];
}
# define directories
# add to parameters' hash
$param{'INDIR'} = $param{'PROJECTNAME'}.'/input';
$param{'OUTDIR'} = $param{'PROJECTNAME'}.'/output';
.... do something ...
# #samples is a list of sample names
foreach (#samples)
{
# for each sample, pass the hash values & sample name to a separate script
system('perl script2.pl <hash> $_');
}
script2.pl
#!/usr/bin/perl -w
use Data::Dumper;
## usage <script2.pl> <hash> <samplename>
# something like getting and printing the hash
my #string = $ARGV[0];
print #string;
If you can help me showing how to pass and get the hash object (something simple like printing the hash object in the second script would do), then I'd appreciate your help.
Thanks!
What you're looking for is something called serialisation. It's difficult to directly represent a memory structure in such a way as to pass it between processes, because of all sorts of fun things like pointers and buffers.
So you need to turn your hash into something simple enough to hand over in a single go.
Three key options for this in my opinion:
Storable - a perl core module that lets you freeze and thaw a data structure for this sort of purpose.
JSON - a text based representation of a hash-like structure.
XML - bit like JSON, but with slightly different strengths/weaknesses.
Which you should use depends a little on how big your data structure is.
Storable is probably the simplest, but it's not going to be particularly portable.
There's also Data::Dumper that's an option too, as it prints data structures. Generally though, I'd suggest that has all the downsides of all the above - you still need to parse it like JSON/XML but it's also not portable.
Example using Storable:
use strict;
use warnings;
use Storable qw ( freeze );
use MIME::Base64;
my %test_hash = (
"fish" => "paste",
"apples" => "pears"
);
my $frozen = encode_base64 freeze( \%test_hash );
system( "perl", "some_other_script.pl", $frozen );
Calling:
use strict;
use warnings;
use Storable qw ( thaw );
use Data::Dumper;
use MIME::Base64;
my ($imported_scalar) = #ARGV;
print $imported_scalar;
my $thing = thaw (decode_base64 $imported_scalar ) ;
print Dumper $thing;
Or:
my %param = %{ thaw (decode_base64 $imported_scalar ) };
print Dumper \%param;
This will print:
BAoIMTIzNDU2NzgEBAQIAwIAAAAKBXBhc3RlBAAAAGZpc2gKBXBlYXJzBgAAAGFwcGxlcw==
$VAR1 = {
'apples' => 'pears',
'fish' => 'paste'
};
Doing the same with JSON - which has the advantage of being passed as plain text, and in a general purpose format. (Most languages can parse JSON):
#!/usr/bin/env perl
use strict;
use warnings;
use JSON;
my %test_hash = (
"fish" => "paste",
"apples" => "pears"
);
my $json_text = encode_json ( \%test_hash );
print "Encoded: ",$json_text,"\n";
system( "perl", "some_other_script.pl", quotemeta $json_text );
Calling:
#!/usr/bin/env perl
use strict;
use warnings;
use JSON;
use Data::Dumper;
my ($imported_scalar) = #ARGV;
$imported_scalar =~ s,\\,,g;
print "Got: ",$imported_scalar,"\n";
my $thing = decode_json $imported_scalar ;
print Dumper $thing;
Need the quotemeta and the removal of slashes unfortunately, because the shell interpolates them. This is the common problem if you're trying to do this sort of thing.

How to read back a file written using Data::Dumper (but not with the default VAR naming)?

I have written a hash structure into a file using
print FILE Data::Dumper->Dump([$x], [ qw(*x) ]);
How do I read this back from file? If I use eval as shown in the following snippet, all I get is $VAR1 = undef;
local $/; $hash_ref = eval <FILE>;
You can use the core module Storable to do this type of task.
use Storable;
store \%table, 'file';
$hashref = retrieve('file');
If those two statements are in the same file, then you can't do it that way unless you close the file handle and reopen the same file for reading. You could do something fancy like opening it with mode +> and then using seek to get back to the beginning before you read again, but why bother, especially since you already have a variable in the same program that contains the same data.
So I assume you are dumping the data from one program and reading it again in another. The problem with using Data::Dumper->Dump([$x], ['*x']) is that it will dereference $x and invent a variable of the appropriate type with the given name. So if $x is a hash reference it will name the variable %x, if it is an array reference then it will be #x etc.
It is far better to remove the star and just write Data::Dumper->Dump([$x], ['x']), which will avoid the dereferencing and name the variable $x.
To read the file back in you should just use do. Like this
use strict;
use warnings;
use Data::Dumper;
my $x = {
a => 1,
b => 2,
};
open my $fh, '>', 'dumper.txt' or die $!;
print $fh Data::Dumper->Dump([$x], ['x']);
close $fh;
my $data = do 'dumper.txt';
If you are constrained to using the form of Data::Dumper call that you show, then you must provide a variable of the appropriate type, like this
use strict;
use warnings;
use Data::Dumper;
my $x = {
a => 1,
b => 2,
};
open my $fh, '>', 'dumper.txt' or die $!;
print $fh Data::Dumper->Dump([$x], ['*x']);
close $fh;
my %data = do 'dumper.txt';
Note that, although the Data::Dumper output file refers to a variable %x, the file is run as a separate Perl program and there is no %x in the program that executes the do.
I tried several ways to export an existing hash. The only way I found that works is to create a new var that is a pointer to the existing hash.
Then Borodin's answer works well.
use strict;
use warnings;
use Data::Dumper;
my %x = (
a => 1,
b => 2,
);
my $x = \%x; # <<< Added so $x is a reference to %x.
open my $fh, '>', 'dumper.txt' or die $!;
print $fh Data::Dumper->Dump([$x], ['*x']);
close $fh;
my %data = do 'dumper.txt';

Passing a 2D array from one Perl script to another

I have a Perl script called master.pl. I have a 2D array in that called #inputarray.
I need to pass the 2D array values from master.pl to another program called child.pl and access the data in the child.pl.
I have tried a lot but I'm not able to dereference the array in child.pl.
Can you please help me?
master.pl
system "start perl child.pl $appl $count #inputarray";
child.pl
($appl, $count, #inputarray) = #ARGV;
for (my $k = 0; $k < $count + 1; $k++) {
for (my $m = 0; $m < 6; $m++) {
print "$inputarray[$k][$m] ";
}
print "\n";
}
Method 1:
Have a look at the standard module Data::Dumper, it's ideal for what you want.
Save your data structure in a temporary file using Data::Dumper and then read it in your second script.
Method 2:
Using Storable to store array in first script and retrieve it from other.
Edit (after you provided your code):
See you can access the array like this
master.pl
#!/usr/local/bin/perl
use strict;
use warnings;
use Storable;
my #inputarray = ([1, 2, 3], [4, 5, 6], [7, 8, 9]);
store (\#inputarray, "/home/chankey/child.$$") or die "could not store";
system("perl", "child.pl", $$) == 0 or die "error";
child.pl
#/usr/local/bin/perl
use strict;
use warnings;
use Storable;
use Data::Dumper;
my $parentpid = shift;
my $ref = retrieve("/home/chankey/child.$parentpid") or die "coudn't retrieve";
print Dumper $ref;
print $$ref[0][0]; #prints 1
Output
$VAR1 = [
[
1,
2,
3
],
[
4,
5,
6
],
[
7,
8,
9
]
]; #from Dumper
1 #from print $$ref[0][0]
As you can see from dump, you've received the #inputarray in $ref. Now use it the way you want.
Check Storable for any kind of perl data serialization/deserialization.
For passing data between processes on POSIX systems I use named pipes, for better compatibility with Windows you could use temporary files, using File::Temp.
You could use an anonymous pipe, which works in the same way on UNIX and Windows (I assume you are using Windows because of the start.
Try this:
use strict;
use warnings;
my $appl = 'orange';
my #inputarray = ([0,1,2],[3,4,5],[6,7,8]);
We don't need $count, you can get the number of elements in the array using scalar context, or the highest index number using $#inputarray;
I have omitted the start because it makes it difficult to debug (the console window closes after the run).
my $cmd = 'perl child.pl';
open(my $pipe, '|-', $cmd) or
die "Unable to execte $cmd; $!";
Using Data::Dumper we can add eval statements and reduce whitespace generation:
use Data::Dumper;
local $Data::Dumper::Purity = 1;
local $Data::Dumper::Indent = 0;
my $dat = Data::Dumper->new([\$appl,\#inputarray],
[qw($appl $inputarray)]);
print $pipe $dat->Dump();
close ($pipe);
Now for the child, which reads the pipe (input stream):
use strict;
use warnings;
my ($inp) = <STDIN>;
my ($appl, $inputarray);
eval "$inp";
print "appl = $$appl\n";
Use of eval is usually frowned upon, and it can give a security hole, so use with care. I think it is justifed here.
Your loops are a tad complex, and smell of C. These are rather more Perlish:
for my $ref (#$inputarray) {
for my $ele (#$ref) {
print "$ele "
}
print "\n"
}
YAML is safer, because it does not require the eval, but needs to be installed.