I have a large multi-dimensional hash which is an import of a JSON structure.
my %bighash;
There is an element in %bighash called:
$bighash{'core'}{'dates'}{'year'} = 2019.
I have a separate string variable called core.dates.year which I would like to use to extract 2019 from %bighash.
I've written this code:
my #keys = split(/\./, 'core.dates.year');
my %hash = ();
my $hash_ref = \%hash;
for my $key ( #keys ){
$hash_ref->{$key} = {};
$hash_ref = $hash_ref->{$key};
}
which when I execute:
say Dumper \%hash;
outputs:
$VAR1 = {
'core' => {
'dates' => {
'year' => {}
}
}
};
All good so far. But what I now want to do is say:
print $bighash{\%hash};
Which I want to return 2019. But nothing is being returned or I'm seeing an error about "Use of uninitialized value within %bighash in concatenation (.) or string at script.pl line 1371, line 17 (#1)...
Can someone point me into what is going on?
My project involves embedding strings in an external file which is then replaced with actual values from %bighash so it's just string interpolation.
Thanks!
Can someone point me into what is going on [when I use $bighash{\%hash}]?
Hash keys are strings, and the stringification of \%hash is something like HASH(0x655178). The only element in %bighash has core —not HASH(0x655178)— for key, so the hash lookup returns undef.
Useful tools:
sub dive_val :lvalue { my $p = \shift; $p //= \( $$p->{$_} ) for #_; $$p } # For setting
sub dive { my $r = shift; $r //= $r->{$_} for #_; $r } # For getting
dive_val(\%hash, split /\./, 'core.dates.year') = 2019;
say dive(\%hash, split /\./, 'core.dates.year');
Hash::Fold would seem to be helpful here. You can "flatten" your hash and then access everything with a single key.
use Hash::Fold 'flatten';
my $flathash = flatten(\%bighash, delimiter => '.');
print $flathash->{"core.dates.year"};
There are no multi-dimensional hashes in Perl. Hashes are key/value pairs. Your understanding of Perl data structures is incomplete.
Re-imagine your data structure as follows
my %bighash = (
core => {
dates => {
year => 2019,
},
},
);
There is a difference between the round parentheses () and the curly braces {}. The % sigil on the variable name indicates that it's a hash, that is a set of unordered key/value pairs. The round () are a list. Inside that list are two scalar values, i.e. a key and a value. The value is a reference to another, anonymous, hash. That's why it has curly {}.
Each of those levels is a separate, distinct data structure.
This rewrite of your code is similar to what ikegami wrote in his answer, but less efficient and more verbose.
my #keys = split( /\./, 'core.dates.year' );
my $value = \%bighash;
for my $key (#keys) {
$value //= $value->{$key};
}
print $value;
It drills down step by step into the structure and eventually gives you the final value.
Related
I'm new to using perl and I'm trying to build a hash of a hash from a tsv. My current process is to read in a file and construct a hash and then insert it into another hash.
my %hoh = ();
while (my $line = <$tsv>)
{
chomp $line;
my %hash;
my #data = split "\t", $line;
my $id;
my $iter = each_array(#columns, #data);
while(my($k, $v) = $iter->())
{
$hash{$k} = $v;
if($k eq 'Id')
{
$id = $v;
}
}
$hoh{$id} = %hash;
}
print "dump: ", Dumper(%hoh);
This outputs:
dump
$VAR1 = '1234567890';
$VAR2 = '17/32';
$VAR3 = '1234567891';
$VAR4 = '17/32';
.....
Instead of what I would expect:
dump
{
'1234567890' => {
'k1' => 'v1',
'k2' => 'v2',
'k3' => 'v3',
'k4' => 'v4',
'id' => '1234567890'
},
'1234567891' => {
'k1' => 'v1',
'k2' => 'v2',
'k3' => 'v3',
'k4' => 'v4',
'id' => '1234567891'
},
........
};
My limited understanding is that when I do $hoh{$id} = %hash; its inserting in a reference to %hash? What am I doing wrong? Also is there a more succint way to use my columns and data array's as key,value pairs into my %hash object?
-Thanks in advance,
Niru
To get a reference, you have to use \:
$hoh{$id} = \%hash;
%hash is the hash, not the reference to it. In scalar context, it returns the string X/Y wre X is the number of used buckets and Y the number of all the buckets in the hash (i.e. nothing useful).
To get a reference to a hash variable, you need to use \%hash (as choroba said).
A more succinct way to assign values to columns is to assign to a hash slice, like this:
my %hoh = ();
while (my $line = <$tsv>)
{
chomp $line;
my %hash;
#hash{#columns} = split "\t", $line;
$hoh{$hash{Id}} = \%hash;
}
print "dump: ", Dumper(\%hoh);
A hash slice (#hash{#columns}) means essentially the same thing as ($hash{$columns[0]}, $hash{$columns[1]}, $hash{$columns[2]}, ...) up to however many columns you have. By assigning to it, I'm assigning the first value from split to $hash{$columns[0]}, the second value to $hash{$columns[1]}, and so on. It does exactly the same thing as your while ... $iter loop, just without the explicit loop (and it doesn't extract the $id).
There's no need to compare each $k to 'Id' inside a loop; just store it in the hash as a normal field and extract it afterwards with $hash{Id}. (Aside: Is your column header Id or id? You use Id in your loop, but id in your expected output.)
If you don't want to keep the Id field in the individual entries, you could use delete (which removes the key from the hash and returns the value):
$hoh{delete $hash{Id}} = \%hash;
Take a look at the documentation included in Perl. The command perldoc is very helpful. You can also look at the Perldoc webpage too.
One of the tutorials is a tutorial on Perl references. It all help clarify a lot of your questions and explain about referencing and dereferencing.
I also recommend that you look at CPAN. This is an archive of various Perl modules that can do many various tasks. Look at Text::CSV. This module will do exactly what you want, and even though it says "CSV", it works with tab separated files too.
You missed putting a slash in front of your hash you're trying to make a reference. You have:
$hoh{$id} = %hash;
Probably want:
$hoh{$id} = \%hash;
also, when you do a Data::Dumper of a hash, you should do it on a reference to a hash. Internally, hashes and arrays have similar structures when a Data::Dumper dump is done.
You have:
print "dump: ", Dumper(%hoh);
You should have:
print "dump: ", Dumper( \%hoh );
My attempt at the program:
#! /usr/bin/env perl
#
use warnings;
use strict;
use autodie;
use feature qw(say);
use Data::Dumper;
use constant {
FILE => "test.txt",
};
open my $fh, "<", FILE;
#
# First line with headers
#
my $line = <$fh>;
chomp $line;
my #headers = split /\t/, $line;
my %hash_of_hashes;
#
# Rest of file
#
while ( my $line = <$fh> ) {
chomp $line;
my %line_hash;
my #values = split /\t/, $line;
for my $index ( ( 0..$#values ) ) {
$line_hash{ $headers[$index] } = $values[ $index ];
}
$hash_of_hashes{ $line_hash{id} } = \%line_hash;
}
say Dumper \%hash_of_hashes;
You should only store a reference to a variable if you do so in the last line before the variable goes go of scope. In your script, you declare %hash inside the while loop, so placing this statement as the last in the loop is safe:
$hoh{$id} = \%hash;
If it's not the last statement (or you're not sure it's safe), create an anonymous structure to hold the contents of the variable:
$hoh{$id} = { %hash };
This makes a copy of %hash, which is slower, but any subsequent changes to it will not effect what you stored.
I want to make a subroutine that adds elements (keys with values) to a previously-defined hash. This subroutine is called in a loop, so the hash grows. I don’t want the returning hash to overwrite existing elements.
At the end, I would like to output the whole accumulated hash.
Right now it doesn’t print anything. The final hash looks empty, but that shouldn’t be.
I’ve tried it with hash references, but it does not really work. In a short form, my code looks as follows:
sub main{
my %hash;
%hash=("hello"=>1); # entry for testing
my $counter=0;
while($counter>5){
my(#var, $hash)=analyse($one, $two, \%hash);
print ref($hash);
# try to dereference the returning hash reference,
# but the error msg says: its not an reference ...
# in my file this is line 82
%hash=%{$hash};
$counter++;
}
# here trying to print the final hash
print "hash:", map { "$_ => $hash{$_}\n" } keys %hash;
}
sub analyse{
my $one=shift;
my $two=shift;
my %hash=%{shift #_};
my #array; # gets filled some where here and will be returned later
# adding elements to %hash here as in
$hash{"j"} = 2; #used for testing if it works
# test here whether the key already exists or
# otherwise add it to the hash
return (#array, \%hash);
}
but this doesn’t work at all: the subroutine analyse receives the hash, but its returned hash reference is empty or I don’t know. At the end nothing got printed.
First it said, it's not a reference, now it says:
Can't use an undefined value as a HASH reference
at C:/Users/workspace/Perl_projekt/Extractor.pm line 82.
Where is my mistake?
I am thankful for any advice.
Arrays get flattened in perl, so your hashref gets slurped into #var.
Try something like this:
my ($array_ref, $hash_ref) = analyze(...)
sub analyze {
...
return (\#array, \#hash);
}
If you pass the hash by reference (as you're doing), you needn't return it as a subroutine return value. Your manipulation of the hash in the subroutine will stick.
my %h = ( test0 => 0 );
foreach my $i ( 1..5 ) {
do_something($i, \%h);
}
print "$k = $v\n" while ( my ($k,$v) = each %h );
sub do_something {
my $num = shift;
my $hash = shift;
$hash->{"test${num}"} = $num; # note the use of the -> deference operator
}
Your use of the #array inside the subroutine will need a separate question :-)
foreach my %hash (%myhash1,%myhash2,%myhash3)
{
while (($keys,$$value) = each %hash)
{
#use key and value...
}
}
Why doesn't this work :
it says synta error on foreach line.
Pls tell me why is it wrong.
This is wrong because you seem to think that this allows you to access each hash as a separate construct, whereas what you are in fact doing is, besides a syntax error, accessing the hashes as a mixed-together new list. For example:
my %hash1 = qw(foo 1 bar 1);
my %hash2 = qw(abc 1 def 1);
for (%hash1, %hash2) # this list is now qw(foo 1 bar 1 abc 1 def 1)
When you place a hash (or array) in a list context statement, they are expanded into their elements, and their integrity is not preserved. Some built-in functions do allow this behaviour, but normal Perl code does not.
You also cannot assign a hash as the for iterator variable, that can only ever be a scalar value. What you can do is this:
for my $hash (\%myhash1, \%myhash2, \%myhash3) {
while (my ($key, $value) = each %$hash) {
...
Which is to say, you create a list of hash references and iterate over them. Note that you cannot tell the difference between the hashes with this approach.
Note also that I use my $hash because this variable must be a scalar.
The syntax should be like:
my $hash1 = {'a'=>1};
my $hash2 = {'b'=>1};
my #arr2 = ($hash1, $hash2);
foreach $hash (#arr2)
{
while(($key, $value) = each %$hash)
{
print $key, $value;
}
}
you need to reference and then dereference the hash.
What is the best practise to solve this?
if (... )
{
push (#{$hash{'key'}}, #array ) ;
}
else
{
$hash{'key'} ="";
}
Is that bad practise for storing one element is array or one is just double quote in hash?
I'm not sure I understand your question, but I'll answer it literally as asked for now...
my #array = (1, 2, 3, 4);
my $arrayRef = \#array; # alternatively: my $arrayRef = [1, 2, 3, 4];
my %hash;
$hash{'key'} = $arrayRef; # or again: $hash{'key'} = [1, 2, 3, 4]; or $hash{'key'} = \#array;
The crux of the problem is that arrays or hashes take scalar values... so you need to take a reference to your array or hash and use that as the value.
See perlref and perlreftut for more information.
EDIT: Yes, you can add empty strings as values for some keys and references (to arrays or hashes, or even scalars, typeglobs/filehandles, or other scalars. Either way) for other keys. They're all still scalars.
You'll want to look at the ref function for figuring out how to disambiguate between the reference types and normal scalars.
It's probably simpler to use explicit array references:
my $arr_ref = \#array;
$hash{'key'} = $arr_ref;
Actually, doing the above and using push result in the same data structure:
my #array = qw/ one two three four five /;
my $arr_ref = \#array;
my %hash;
my %hash2;
$hash{'key'} = $arr_ref;
print Dumper \%hash;
push #{$hash2{'key'}}, #array;
print Dumper \%hash2;
This gives:
$VAR1 = {
'key' => [
'one',
'two',
'three',
'four',
'five'
]
};
$VAR1 = {
'key' => [
'one',
'two',
'three',
'four',
'five'
]
};
Using explicit array references uses fewer characters and is easier to read than the push #{$hash{'key'}}, #array construct, IMO.
Edit: For your else{} block, it's probably less than ideal to assign an empty string. It would be a lot easier to just skip the if-else construct and, later on when you're accessing values in the hash, to do a if( defined( $hash{'key'} ) ) check. That's a lot closer to standard Perl idiom, and you don't waste memory storing empty strings in your hash.
Instead, you'll have to use ref() to find out what kind of data you have in your value, and that is less clear than just doing a defined-ness check.
I'm not sure what your goal is, but there are several things to consider.
First, if you are going to store an array, do you want to store a reference to the original value or a copy of the original values? In either case, I prefer to avoid the dereferencing syntax and take references when I can:
$hash{key} = \#array; # just a reference
use Clone; # or a similar module
$hash{key} = clone( \#array );
Next, do you want to add to the values that exist already, even if it's a single value? If you are going to have array values, I'd make all the values arrays even if you have a single element. Then you don't have to decide what to do and you remove a special case:
$hash{key} = [] unless defined $hash{key};
push #{ $hash{key} }, #values;
That might be your "best practice" answer, which is often the technique that removes as many special cases and extra logic as possible. When I do this sort of thing in a module, I typically have a add_value method that encapsulates this magic where I don't have to see it or type it more than once.
If you already have a non-reference value in the hash key, that's easy to fix too:
if( defined $hash{key} and ! ref $hash{key} ) {
$hash{key} = [ $hash{key} ];
}
If you already have non-array reference values that you want to be in the array, you do something similar. Maybe you want an anonymous hash to be one of the array elements:
if( defined $hash{key} and ref $hash{key} eq ref {} ) {
$hash{key} = [ $hash{key} ];
}
Dealing with the revised notation:
if (... )
{
push (#{$hash{'key'}}, #array);
}
else
{
$hash{'key'} = "";
}
we can immediately tell that you are not following the standard advice that protects novices (and experts!) from their own mistakes. You're using a symbolic reference, which is not a good idea.
use strict;
use warnings;
my %hash = ( key => "value" );
my #array = ( 1, "abc", 2 );
my #value = ( 22, 23, 24 );
push(#{$hash{'key'}}, #array);
foreach my $key (sort keys %hash) { print "$key = $hash{$key}\n"; }
foreach my $value (#array) { print "array $value\n"; }
foreach my $value (#value) { print "value $value\n"; }
This does not run:
Can't use string ("value") as an ARRAY ref while "strict refs" in use at xx.pl line 8.
I'm not sure I can work out what you were trying to achieve. Even if you remove the 'use strict;' warning, the code shown does not detect a change from the push operation.
use warnings;
my %hash = ( key => "value" );
my #array = ( 1, "abc", 2 );
my #value = ( 22, 23, 24 );
push #{$hash{'key'}}, #array;
foreach my $key (sort keys %hash) { print "$key = $hash{$key}\n"; }
foreach my $value (#array) { print "array $value\n"; }
foreach my $value (#value) { print "value $value\n"; }
foreach my $value (#{$hash{'key'}}) { print "h_key $value\n"; }
push #value, #array;
foreach my $key (sort keys %hash) { print "$key = $hash{$key}\n"; }
foreach my $value (#array) { print "array $value\n"; }
foreach my $value (#value) { print "value $value\n"; }
Output:
key = value
array 1
array abc
array 2
value 22
value 23
value 24
h_key 1
h_key abc
h_key 2
key = value
array 1
array abc
array 2
value 22
value 23
value 24
value 1
value abc
value 2
I'm not sure what is going on there.
If your problem is how do you replace a empty string value you had stored before with an array onto which you can push your values, this might be the best way to do it:
if ( ... ) {
my $r = \$hash{ $key }; # $hash{ $key } autoviv-ed
$$r = [] unless ref $$r;
push #$$r, #values;
}
else {
$hash{ $key } = "";
}
I avoid multiple hash look-ups by saving a copy of the auto-vivified slot.
Note the code relies on a scalar or an array being the entire universe of things stored in %hash.
I have a function that takes a variable and an associative array, but I can't seem to get them to pass right. I think this has something to do with function declarations, however I can't figure out how they work in Perl. Is there a good reference for this and how do I accomplish what I need?
I should add that it needs to be passed by reference.
sub PrintAA
{
my $test = shift;
my %aa = shift;
print $test . "\n";
foreach (keys %aa)
{
print $_ . " : " . $aa{$_} . "\n";
$aa{$_} = $aa{$_} . "+";
}
}
Pass the reference instead of the hash itself. As in
PrintAA("abc", \%fooHash);
sub PrintAA
{
my $test = shift;
my $aaRef = shift;
print $test, "\n";
foreach (keys %{$aaRef})
{
print $_, " : ", $aaRef->{$_}, "\n";
}
}
See also perlfaq7: How can I pass/return a {Function, FileHandle, Array, Hash, Method, Regex}?
This code works:
#!/bin/perl -w
use strict;
sub PrintAA
{
my($test, %aa) = #_;
print $test . "\n";
foreach (keys %aa)
{
print $_ . " : " . $aa{$_} . "\n";
}
}
my(%hash) = ( 'aaa' => 1, 'bbb' => 'balls', 'ccc' => \&PrintAA );
PrintAA("test", %hash);
The key point is the use of the array context in the my() 'statement' in the function.
What does the array context business actually do?
Succinctly, it makes it work correctly.
It means that the first value in the #_ array of arguments is assigned to $test, and the remaining items are assigned to the hash %aa. Given the way I called it, there is an odd number of items in the #_, so once the first item is assigned to $test, there is an even number of items available to assign to %aa, with the first item of each pair being the key ('aaa', 'bbb', 'ccc' in my example), and the second being the corresponding value.
It would be possible to replace %aa with #aa, in which case, the array would have 6 items in it. It would also be possible to replace %aa with $aa, and in that case, the variable $aa would contain the value 'aaa', and the remaining values in #_ would be ignored by the assignment.
If you omit the parentheses around the variable list, Perl refuses to compile the code.
One of the alternative answers showed the notation:
my $test = shift;
my(%aa) = #_;
This is pretty much equivalent to what I wrote; the difference is that after the two my statements, #_ only contains 6 elements in this variation, whereas in the single my version, it still contains 7 elements.
There are definitely other questions in SO about array context.
Actually, I wasn't asking about the my($test, %aa) = #_; I was asking about my(%hash) = ( 'aaa' => 1, 'bbb' => 'balls', 'ccc' => \&PrintAA ); versus my %hash = { 'aaa' => 1, ... };
The difference is that the { ... } notation generates a hash ref and the ( ... ) notation generates a list, which maps to a hash (as opposed to hash ref). Similarly, [ ... ] generates an array ref and not an array.
Indeed, change the 'main' code so it reads: my(%hash) = { ... }; and you get a run-time (but not compile time) error - treat line numbers with caution since I've added alternative codings to my file:
Reference found where even-sized list expected at xx.pl line 18.
...
Use of uninitialized value in concatenation (.) or string at xx.pl line 13.
Alternatively:
sub PrintAA
{
my $test = shift;
my %aa = #_;
print $test . "\n";
foreach (keys %aa)
{
print $_ . " : " . $aa{$_} . "\n";
$aa{$_} = $aa{$_} . "+";
}
}
The thing you're fundamentally missing is that an associative array isn't a single argument (though an associative array reference is, as in Paul Tomblin's answer).
It looks like you should pass in a reference to a hash.
sub PrintAA
{
my $test = shift;
my $aa = shift;
if (ref($aa) != "HASH") { die "bad arg!" }
....
}
PrintAA($foo, \%bar);
The reason you can't do a
my %aa = shift;
is because Perl flattens all the arguments to a subroutine into one list, #_. Every element is copied, so passing in by reference avoids those copies as well.
As usual there are several ways. Here is what Perl Best Practices, that most revered of style pointers, has to say about passing parameters to functions:
Use a hash of named arguments for any subroutine that has more than three parameters
But since you have only two, you could get away ;) with passing them directly like this:
my $scalar = 5;
my %hash = (a => 1, b => 2, c => 3);
func($scalar, %hash)
And function is defined like this:
sub func {
my $scalar_var = shift;
my %hash_var = #_;
... Do something ...
}
It could be more useful if you could show some code.
All the methods in previous answers work, but this was always the way I preferred to do things like this:
sub PrintAA ($\%)
{
my $test = shift;
my %aa = ${shift()};
print "$test\n";
foreach (keys %aa)
{
print "$_ : $aa{$_}\n";
$aa{$_} = "$aa{$_}+";
}
}
Note: I also changed your code a bit. Perl's double-quoted strings will interpret "$test" to be the value of $test rather than the actual string '$test', so you don't need that many .s.
Also, I was wrong about how the prototypes work. To pass a hash, use this:
PrintAA("test", %hash);
To print a hash reference, use this:
PrintAA("test", %$ref_to_hash);
Of course, now you can't modify the hash referenced by $ref_to_hash because you're sending a copy, but you can modify a raw %hash because you're passing it as a reference.
Arguments to functions get flattened into a single array (#_). So it's usually easiest to pass hashes to a function by reference.
To create a hash:
my %myhash = ( key1 => "val1", key2 => "val2" );
To create a reference to that hash:
my $href = \%myhash
To access that hash by reference;
%$href
So in your sub:
my $myhref = shift;
keys %$myhref;
All the other replies here so far seem rather complicated to me. When I write Perl function I usually "expand" all the passed arguments in the first line of the function.
sub someFunction {
my ( $arg1, $arg2, $arg3 ) = #_;
This is similar to other languages, where you declare functions as
... someFunction ( arg1, arg2, arg3 )
And if you do it that way and pass the hash as the last argument, you'll be fine without any tricks or special magic. E.g.:
sub testFunc {
my ( $string, %hash ) = #_;
print "$string $hash{'abc'} $hash{'efg'} $string\n";
}
my %testHash = (
'abc' => "Hello,",
'efg' => "World!"
);
testFunc('!!!', %testHash);
The output is as expected:
!!! Hello, World! !!!
This works because in Perl arguments are always passed as an array of scalar values and if you pass a hash, its key value/pairs are added to that array. In the sample above, the arguments passed to the function as array (#_) are in fact:
'!!!', 'abc', 'Hello,', 'efg', 'World!'
and '!!!' is simply assigned to %string, while %hash "swallows" all the other arguments, always interpreting one as a key and the next one as value (until all elements are used up).
You cannot pass multiple hashes that way and the hash cannot be the first argument, as otherwise it would swallow everything and leave all other arguments unassigned.
Of course, exactly the same works for array as a last argument. The only difference here is that arrays don't distinguish between keys and values. For them, all arguments left over are values and just get pushed to the array.
Use the following sub to get the hash or hashref - whatever is passed :)
sub get_args { ref( $_[0] ) ? shift() : ( #_ % 2 ) ? {} : {#_}; }
sub PrintAA
{
my $test = shift;
my $aa = get_args(#_);;
# Then
$aa->{somearg} # Do something
$aa->{anotherearg} # Do something
}
Call your function like this:
printAA($firstarg,somearg=>1, anotherarg=>2)
Or like this (no matter):
printAA($firstarg, {somearg=>1, anotherarg=>2})
Or even like this (no matter):
my(%hash) = ( 'aaa' => 1, 'bbb' => 'balls', 'ccc' => \PrintAA );
PrintAA("test", %hash);