sum hash of hash values using perl - perl

I have a Perl script that parses an Excel file and does the following : It counts for each value in column A, the number of elements it has in column B, the script looks like this :
use strict;
use warnings;
use Spreadsheet::XLSX;
use Data::Dumper;
use List::Util qw( sum );
my $col1 = 0;
my %hash;
my $excel = Spreadsheet::XLSX->new('inout_chartdata_ronald.xlsx');
my $sheet = ${ $excel->{Worksheet} }[0];
$sheet->{MaxRow} ||= $sheet->{MinRow};
my $count = 0;
# Iterate through each row
foreach my $row ( $sheet->{MinRow}+1 .. $sheet->{MaxRow} ) {
# The cell in column 1
my $cell = $sheet->{Cells}[$row][$col1];
if ($cell) {
# The adjacent cell in column 2
my $adjacentCell = $sheet->{Cells}[$row][ $col1 + 1 ];
# Use a hash of hashes
$hash{ $cell->{Val} }{ $adjacentCell->{Val} }++;
}
}
print "\n", Dumper \%hash;
The output looks like this :
$VAR1 = {
'13' => {
'klm' => 1,
'hij' => 2,
'lkm' => 4,
},
'12' => {
'abc' => 2,
'efg' => 2
}
};
This works great, my question is : How can I access the elements of this output $VAR1 in order to do : for value 13, klm + hij = 3 and get a final output like this :
$VAR1 = {
'13' => {
'somename' => 3,
'lkm' => 4,
},
'12' => {
'abc' => 2,
'efg' => 2
}
};
So basically what I want to do is loop through my final hash of hashes and access its specific elements based on a unique key and finally do their sum.
Any help would be appreciated.
Thanks

I used #do_sum to indicate what changes you want to make. The new key is hardcoded in the script. Note that the new key is not created if no key exists in the subhash (the $found flag).
#!/usr/bin/perl
use warnings;
use strict;
use Data::Dumper;
my %hash = (
'13' => {
'klm' => 1,
'hij' => 2,
'lkm' => 4,
},
'12' => {
'abc' => 2,
'efg' => 2
}
);
my #do_sum = qw(klm hij);
for my $num (keys %hash) {
my $found;
my $sum = 0;
for my $key (#do_sum) {
next unless exists $hash{$num}{$key};
$sum += $hash{$num}{$key};
delete $hash{$num}{$key};
$found = 1;
}
$hash{$num}{somename} = $sum if $found;
}
print Dumper \%hash;

It sounds like you need to learn about Perl References, and maybe Perl Objects which are just a nice way to deal with references.
As you know, Perl has three basic data-structures:
Scalars ($foo)
Arrays (#foo)
Hashes (%foo)
The problem is that these data structures can only contain scalar data. That is, each element in an array can hold a single value or each key in a hash can hold a single value.
In your case %hash is a Hash where each entry in the hash references another hash. For example:
Your %hash has an entry in it with a key of 13. This doesn't contain a scalar value, but a references to another hash with three keys in it: klm, hij, and lkm. YOu can reference this via this syntax:
${ hash{13} }{klm} = 1
${ hash{13} }{hij} = 2
${ hash{13} }{lkm} = 4
The curly braces may or may not be necessary. However, %{ hash{13} } references that hash contained in $hash{13}, so I can now reference the keys of that hash. You can imagine this getting more complex as you talk about hashes of hashes of arrays of hashes of arrays. Fortunately, Perl includes an easier syntax:
$hash{13}->{klm} = 1
%hash{13}->{hij} = 2
%hash{13}->{lkm} = 4
Read up about hashes and how to manipulate them. After you get comfortable with this, you can start working on learning about Object Oriented Perl which handles references in a safer manner.

Related

What does it mean to have a perl hash{}{}

My professor has some syntax on a slide that I do not understand.
In perl there is:
$hash{$string}{$anotherString}++;
What does this syntax mean? If it were:
$hash{$string}{$int}++;
Would it be increment the value?
When I print using
while( my( $key, $value ) = each %hash ){print "$key: $value\n";}
My output is
"key": HASH(0xbe0200)
That is a two-dimensional hash, a hash of hashes. It is easy to keep track of structures in Perl once you realize that any single value is in fact a scalar. In the case of multidimensional structures, the scalar value is a reference. For example:
my %outer = ( "foo" => { "bar" => 1 } );
The inner part { "bar" => 1 } is a hash reference. The use of { } in assignment denotes an anonymous hash. This is similar to:
my %inner = ( "bar" => 1 );
my %outer = ( "foo" => \%inner );
Now when you want to reference a value in %inner, you use the first key to access the hash reference, and the second key to access the value in %inner:
print $outer{"foo"}{"bar"}; # prints 1
And when you use the increment operator ++ on a value, it is incremented:
$outer{"foo"}{"bar"}++; # the value is now 2
$hash{string1}{string2}
is a shorter equivalent of
$hash{string1}->{string2}
i.e. it returns a value from a hash of hashes.
By applying the ++ operator, the value in the inner hash is incremented.
My output is "key": HASH(0xbe0200)
That strange output means that what you are trying to print is actually a hash reference:
use strict;
use warnings;
use 5.016; #allows you to use say(), which is equivalent to print()
#plus a newline at the end
my $href = {
a => 1,
b => 2,
};
say $href;
--output:--
HASH(0x100826698)
Or,
my %hash = (
a => 1,
b => 2,
);
say \%hash;
--output:--
HASH(0x1008270a0)
The \ operator gets the reference for the thing on its right hand side.
The easiest way to print the actual hash is using Data::Dumper, which is something you can and will use all the time:
use strict;
use warnings;
use 5.016;
use Data::Dumper;
my $href = {
a => 1,
b => 2,
};
say Dumper($href);
$VAR1 = {
'a' => 1,
'b' => 2
};
Like use warnings;, I consider use Data::Dumper; mandatory for every program.
So, when you see strange output, like HASH(0xbe0200), use Data::Dumper on the value:
my %hash = (
a => 1,
b => { hello => 2, goodbye => 3},
);
while( my( $key, $value ) = each %hash ){
say $key;
say Dumper($value);
say '-' x 10;
}
--output:--
a
$VAR1 = 1;
----------
b
$VAR1 = {
'hello' => 2,
'goodbye' => 3
};
----------
Or, alternatively just use Data::Dumper on the whole structure:
my %hash = (
a => 1,
b => { hello => 2, goodbye => 3},
);
say Dumper(\%hash);
--output:--
$VAR1 = {
'a' => 1,
'b' => {
'hello' => 2,
'goodbye' => 3
}
};
Note that Dumper() is used to show the contents of a hash reference(or any other reference), so if your variable is not a reference, e.g. %hash, then you must turn it into a reference using the \ operator, e.g. \%hash.
Now, if you have this hash:
my %hash = (
a => 1,
b => { hello => 2, goodbye => 3},
);
...to retrieve the value corresponding to 'goodbye', you can write:
say $hash{b}{goodbye}; #=>3
$hash{b} returns the hash (reference) { hello => 2, goodbye => 3}, and you can retrieve values from that hash by using the subscripts {hello} or {goodbye}.
Alternatively, you can write this:
my %hash = (
a => 1,
b => { hello => 2, goodbye => 3},
);
my $string = 'b';
my $anotherString = 'goodbye';
say $hash{$string}{$anotherString}; #=>3
And to increment the value 3 in the hash, you can write:
my $result = $hash{$string}{$anotherString}++;
say $result; #=>3
say $hash{$string}{$anotherString}; #=>4
The postfix ++ operator actually increments the value after the current operation, so $result is 3, then the value in the hash is incremented to 4, something like this:
my $temp = $hash{$string}{$anotherString};
$hash{$string}{$anotherString} = $hash{$string}{$anotherString} + 1;
my $result = $temp;
If you want the increment to happen before the current operation, then you can use the prefix ++ operator:
my $result = ++$hash{$string}{$anotherString};
say $result; #=>4
say $hash{$string}{$anotherString}; #=>4
Finally, if the value at $hash{$string}{$anotherString} is not a number, e.g. 'green', you will get something strange:
my %hash = (
a => 1,
b => { hello => 2, goodbye => 'green'},
);
my $string = 'b';
my $anotherString = 'goodbye';
my $result = $hash{$string}{$anotherString}++;
say $hash{$string}{$anotherString};
--output:--
greeo
perl has a notion that the string that comes after the string 'green' is the string 'greeo' because the letter 'o' comes after the letter 'n' in the alphabet. And if the string you incremented were 'greez' the output would be:
greez original
grefa output
The next letter after 'z' is to start over with 'a', but just like when you increment 9 by 1 and get 10, the increment for 'z' carries over to the column on the left, incrementing that letter by 1, producing the 'f'. Ha!

Adding hash keys

I am adding data to a hash using an incrementing numeric key starting at 0. The key/value is fine. When I add the second one, the first key/value pair points back to the second. Each addition after that replaces the value of the second key and then points back to it. The Dumper output would be something like this.
$VAR1 = { '0' => { ... } };
After the first key/value is added. After the second one is added I get
$VAR1= { '1' => { ... }, '0' => $VAR1->{'1} };
After the third key/value is added, it looks like this.
$VAR1 = { '1' => { ... }, '0' => $VAR1->{'1'}, '2' => $VAR1->{'1'} };
My question is why is it doing this? I want each key/value to show up in the hash. When I iterate through the hash I get the same data for every key/value. How do I get rid of the reference pointers to the second added key?
You are setting the value of every element to a reference to the same hash. Data::Dumper is merely reflecting that.
If you're using Data::Dumper as a serializing tool (yuck!), then you should set $Data::Dumper::Purity to 1 to get something eval can process.
use Data::Dumper qw( Dumper );
my %h2 = (a=>5,b=>6,c=>7);
my %h;
$h{0} = \%h2;
$h{1} = \%h2;
$h{2} = \%h2;
print("$h{0}{c} $h{2}{c}\n");
$h{0}{c} = 9;
print("$h{0}{c} $h{2}{c}\n");
{
local $Data::Dumper::Purity = 1;
print(Dumper(\%h));
}
Output:
7 7
9 9
$VAR1 = {
'0' => {
'c' => 9,
'a' => 5,
'b' => 6
},
'1' => {},
'2' => {}
};
$VAR1->{'0'} = $VAR1->{'1'};
$VAR1->{'2'} = $VAR1->{'1'};
If, on the other hand, you didn't mean to use store references to different hashes, you could use
# Shallow copies
$h{0} = { %h2 }; # { ... } means do { my %anon = ( ... ); \%anon }
$h{1} = { %h2 };
$h{2} = { %h2 };
or
# Deep copies
use Storable qw( dclone );
$h{0} = dclone(\%h2);
$h{1} = dclone(\%h2);
$h{2} = dclone(\%h2);
Output:
7 7
9 7
$VAR1 = {
'0' => {
'a' => 5,
'b' => 6,
'c' => 9
},
'1' => {
'a' => 5,
'b' => 6,
'c' => 7
},
'2' => {
'a' => 5,
'b' => 6,
'c' => 7
}
};
You haven't posted the actual code you're using to build the hash, but I assume it looks something like this:
foreach my $i (1 .. 3) {
%hash2 = (number => $i, foo => "bar", baz => "whatever");
$hash1{$i} = \%hash2;
}
(Actually, I'll guess that, in your actual code, you're probably reading data from a file in a while (<>) loop and assigning values to %hash2 based on it, but the foreach loop will do for demonstration purposes.)
If you run the code above and dump the resulting %hash1 using Data::Dumper, you'll get the output:
$VAR1 = {
'1' => {
'baz' => 'whatever',
'number' => 3,
'foo' => 'bar'
},
'3' => $VAR1->{'1'},
'2' => $VAR1->{'1'}
};
Why does it look like that? Well, it's because the values in %hash1 are all references pointing to the same hash, namely %hash2. When you assign new values to %hash2 in your loop, those values will overwrite the old values in %hash2, but it will still be the same hash. Data::Dumper is just highlighting that fact.
So, how can you fix it? Well, there are (at least) two ways. One way is to replace \%hash2, which gives a reference to %hash2, with { %hash2 }, which copies the contents of %hash2 into a new anonymous hash and returns a reference to that:
foreach my $i (1 .. 3) {
%hash2 = (number => $i, foo => "bar", baz => "whatever");
$hash1{$i} = { %hash2 };
}
The other (IMO preferable) way is to declare %hash2 as a (lexically scoped) local variable within the loop using my:
foreach my $i (1 .. 3) {
my %hash2 = (number => $i, foo => "bar", baz => "whatever");
$hash1{$i} = \%hash2;
}
This way, each iteration of the loop will create a new, different hash named %hash2, while the hashes created on previous iterations will continue to exist (since they're referenced from %hash1) independently.
By the way, you wouldn't have had this problem in the first place if you'd followed standard Perl best practices, specifically:
Always use strict; (and use warnings;). This would've forced you to declare %hash2 with my (although it wouldn't have forced you to do so inside the loop).
Always declare local variables in the smallest possible scope. In this case, since %hash2 is only used within the loop, you should've declared it inside the loop, like above.
Following these best practices, the example code above would look like this:
use strict;
use warnings;
use Data::Dumper qw(Dumper);
my %hash1;
foreach my $i (1 .. 3) {
my %hash2 = (number => $i, foo => "bar", baz => "whatever");
$hash1{$i} = \%hash2;
}
print Dumper(\%hash1);
which, as expected, will print:
$VAR1 = {
'1' => {
'baz' => 'whatever',
'number' => 1,
'foo' => 'bar'
},
'3' => {
'baz' => 'whatever',
'number' => 3,
'foo' => 'bar'
},
'2' => {
'baz' => 'whatever',
'number' => 2,
'foo' => 'bar'
}
};
It's hard to see what the problem is when you don't post the code or the actual results of Data::Dumper.
There is one thing you should know about Data::Dumper: When you dump an array or (especially) a hash, you should dump a reference to it. Otherwise, Data::Dumper will treat it like a series of variables. Also notice that hashes do not remain in the order you create them. I've enclosed an example below. Make sure that your issue isn't related to a confusing Data::Dumper output.
Another question: If you're keying your hash by sequential keys, would you be better off with an array?
If you can, please edit your question to post your code and the ACTUAL results.
use strict;
use warnings;
use autodie;
use feature qw(say);
use Data::Dumper;
my #array = qw(one two three four five);
my %hash = (one => 1, two => 2, three => 3, four => 4);
say "Dumped Array: " . Dumper #array;
say "Dumped Hash: " . Dumper %hash;
say "Dumped Array Reference: " . Dumper \#array;
say "Dumped Hash Reference: " . Dumper \%hash;
The output:
Dumped Array: $VAR1 = 'one';
$VAR2 = 'two';
$VAR3 = 'three';
$VAR4 = 'four';
$VAR5 = 'five';
Dumped Hash: $VAR1 = 'three';
$VAR2 = 3;
$VAR3 = 'one';
$VAR4 = 1;
$VAR5 = 'two';
$VAR6 = 2;
$VAR7 = 'four';
$VAR8 = 4;
Dumped Array Reference: $VAR1 = [
'one',
'two',
'three',
'four',
'five'
];
Dumped Hash Reference: $VAR1 = {
'three' => 3,
'one' => 1,
'two' => 2,
'four' => 4
};
The reason it is doing this is you are giving it the same reference to the same hash.
Presumably in a loop construct.
Here is a simple program which has this behaviour.
use strict;
use warnings;
# always use the above two lines until you
# understand completely why they are recommended
use Data::Printer;
my %hash;
my %inner; # <-- wrong place to put it
for my $index (0..5){
$inner{int rand} = $index; # <- doesn't matter
$hash{$index} = \%inner;
}
p %hash;
To fix it just make sure that you are creating a fresh hash reference every time through the loop.
use strict;
use warnings;
use Data::Printer;
my %hash;
for my $index (0..5){
my %inner; # <-- place the declaration here instead
$inner{int rand} = $index; # <- doesn't matter
$hash{$index} = \%inner;
}
p %hash;
If you are only going to use numbers for your indexes, and they are monotonically increasing starting from 0, then I would recommend using an array.
An array would be faster and more memory efficient.
use strict;
use warnings;
use Data::Printer;
my #array; # <--
for my $index (0..5){
my %inner;
$inner{int rand} = $index;
$array[$index] = \%inner; # <--
}
p #array;

Perl - Hash of Array: Key look-up by Value

I have a hash of arrays (HoA). I have been processing the values of this HoA using $arrayrefs. However, now I need to retrieve the $key based on the $arrayrefs.
my %a = ( 1 => "ONE" ,
2 => "TWO" ,
3 => " Three", );
my %aa = ( 4 => [ 'ONE' , 'TWO', 'THREE'],
5 => ['one' , 'two', 'three'],
6 => ['more', 'dos', 'some'],
);
my #array = ('ONE' , 'TWO', 'THREE');
my $array_ref = \#array;
# returns the $key where the $value is 'ONE'
my ($any_match) = grep { $a{$_} eq 'ONE' } keys %a;
print $any_match."\n"; # this returns '1', as expected.. Good!
my ($match) = grep { $aa{$_} eq #$array_ref } keys %aa;
print $match."\n"; # <--- error: says that match is uninitialized
In the last print statement, I would like it to return 4. Does anyone know how to do this?
You can't compare arrays with eq. A simple solution is to turn both arrays into strings and comparing the strings using eq:
my ($match) = grep { join("", #{$aa{$_}}) eq join("", #$array_ref) } keys %aa;
For comparing arrays you could also utilize one of many modules from CPAN, e.g. Array::Compare, List::Compare, etc.
Always use strict; use warnings;. Add use v5.10; since Perl's (v5.10+) smart matching will be used to compare arrays. Do the following:
my ($match) = grep { #{$aa{$_}} ~~ #$array_ref } keys %aa;
The smart operator ~~ is used here to compare the arrays.

Slicing a nested hash in Perl

Say I have a hash that I can index as:
$hash{$document}{$word}
From what I read online (although I could not find this on perlreftut, perldsc or perllol), I can slice a hash using a list if I use the # prefix on my hash to indicate that I want the hash to return a list. However, if I try to slice my hash using a list #list:
#%hash{$document}{#list}
I get several "Scalar values ... better written" errors.
How can I slash a nested hash in Perl?
The sigill for your hash must be #, like so:
#{$hash{$document}}{#list}
Assuming #list contains valid keys for %hash it will return the corresponding values, or undef if the key does not exist.
This is based on the general rule of a hash slice:
%foo = ( a => 1, b => 2, c => 3 );
print #foo{'a','b'}; # prints 12
%bar = ( foo => \%foo ); # foo is now a reference in %bar
print #{ $bar{foo} }{'a','b'}; # prints 12, same data as before
First, when you expect to get a list from a hash slice, use # sigil first. % is pointless here.
Second, you should understand that $hash{$document} value is not a hash or array. It's a reference - to a hash OR to an array.
With all this said, you might use something like this:
#{ $hash{$document} }{ #list };
... so you dereference value of $hash{$document}, then use a hash slice over it. For example:
my %hash = (
'one' => {
'first' => 1,
'second' => 2,
},
'two' => {
'third' => 3,
'fourth' => 4,
}
);
my $key = 'one';
my #list = ('first', 'second');
print $_, "\n" for #{ $hash{$key} }{#list};
# ...gives 1\n2\n

Perl map - need to map an array into a hash as arrayelement->array_index

I have a array like this:
my #arr = ("Field3","Field1","Field2","Field5","Field4");
Now i use map like below , where /DOSOMETHING/ is the answer am seeking.
my %hash = map {$_ => **/DOSOMETHING/** } #arr
Now I require the hash to look like below:
Field3 => 0
Field1 => 1
Field2 => 2
Field5 => 3
Field4 => 4
Any help?
%hash = map { $arr[$_] => $_ } 0..$#arr;
print Dumper(\%hash)
$VAR1 = {
'Field4' => 4,
'Field2' => 2,
'Field5' => 3,
'Field1' => 1,
'Field3' => 0
};
my %hash;
#hash{#arr} = 0..$#arr;
In Perl 5.12 and later you can use each on an array to iterate over its index/value pairs:
use 5.012;
my %hash;
while(my ($index, $value) = each #arr) {
$hash{$value} = $index;
}
Here's one more way I can think of to accomplish this:
sub get_bumper {
my $i = 0;
sub { $i++ };
}
my $bump = get_bumper; # $bump is a closure with its very own counter
map { $_ => $bump->(); } #arr;
As with many things that you can do in Perl: Don't do this. :) If the sequence of values you need to assign is more complex (e.g. 0, 1, 4, 9, 16... or a sequence of random numbers, or numbers read from a pipe), it's easy to adapt this approach to it, but it's generally even easier to just use unbeli's approach. The only advantage of this method is that it gives you a nice clean way to provide and consume arbitrary lazy sequences of numbers: a function that needs a caller-specified sequence of numbers can just take a coderef as a parameter, and call it repeatedly to get the numbers.
A very old question, but i had the same problem and this is my solution:
use feature ':5.10';
my #arr = ("Field3","Field1","Field2","Field5","Field4");
my %hash = map {state $i = 0; $_ => $i++} #arr;