How to remove duplicate reference in array? - perl

Consider this array:
my #array = (hashref1, hashref2, hashref3, hashref1);
How can I remove the duplicate reference hashref1.

If by "duplicate reference" you mean they both refer to the same anonymous hash, you can use List::Util's uniq:
#!/usr/bin/perl
use warnings;
use strict;
use List::Util qw{ uniq };
use Data::Dumper;
my $hashref1 = {a => 10};
my $hashref2 = {b => 11};
my $hashref3 = {c => 12};
my #array = ($hashref1, $hashref2, $hashref3, $hashref1);
print Dumper(uniq(#array));
Output:
$VAR1 = {
'a' => 10
};
$VAR2 = {
'b' => 11
};
$VAR3 = {
'c' => 12
};
The stringified versions of the references will be used for comparison, i.e. something like HASH(0x560ee5fdf220). The same references have the same address.
But, if you mean they refer to different objects with the same contents, you need to find a way how to stringify the hashrefs so that the contents is always the same.
#!/usr/bin/perl
use warnings;
use strict;
use List::Util qw{ uniq };
use Data::Dumper;
my $hashref1 = {a => 10, A => 2};
my $hashref2 = {b => 11, B => 3};
my $hashref3 = {c => 12, C => 4};
my $hashref4 = {a => 10, A => 2};
my #array = ($hashref1, $hashref2, $hashref3, $hashref4);
my #serialized = do {
# Comment out the following line to see how 1 and 4
# can be different sometimes.
local $Data::Dumper::Sortkeys = 1;
map Dumper($_), #array;
};
my $VAR1;
print Dumper(map eval, uniq(#serialized));

Related

Binding two Perl hashes (not two anonymous hashes)

In order to reuse some already written code libraries, I need to "masquerade" a hash (%bar) into another hash (%foo) so that each changes made in %bar can be seen with %foo after their binding.
With an anonymous hash construct, it's easy: just copy the ref of bar into the ref of foo ( $foo = $bar; ).
How about "not using" an anonymous hash (because the code I have to reuse is built not using anonymous hash constructs) ? Is this only possible ?
Thanks.
use strict;
use Data::Dumper;
my %foo = ();
my %bar = ();
%foo = %bar; # This won't work as it copies %bar into %foo as at the current time.
$bar{A}->{st} = 'a';
$bar{A}->{qt} = 'a';
$bar{B}->{st} = 'b';
$bar{B}->{qt} = 'b';
# $foo{A}->{st} doesn't exist
Of course, an anonymous hash construct would have been a blessing.
use strict;
use Data::Dumper;
my $foo = {};
my $bar = {};
$foo = $bar; # This works fine.
$bar->{A}->{st} = 'a';
$bar->{A}->{qt} = 'a';
$bar->{B}->{st} = 'b';
$bar->{B}->{qt} = 'b';
print STDOUT "foo\n---\n";
print Dumper($foo) . "\n\n";
print STDOUT "bar\n---\n";
print Dumper($bar) . "\n\n";
Yields the expected result:
foo
---
$VAR1 = {
'B' => {
'st' => 'b',
'qt' => 'b'
},
'A' => {
'qt' => 'a',
'st' => 'a'
}
};
bar
---
$VAR1 = {
'B' => {
'st' => 'b',
'qt' => 'b'
},
'A' => {
'qt' => 'a',
'st' => 'a'
}
};
For your example, you can use Data::Alias to alias %bar to %foo. That allows you to make changes in %bar and later see those in %foo.
use strict;
use warnings;
use Data::Printer;
use Data::Alias;
my %foo = ();
my %bar = ();
alias %foo = %bar;
$bar{A}->{st} = 'a';
$bar{A}->{qt} = 'a';
$bar{B}->{st} = 'b';
$bar{B}->{qt} = 'b';
p %foo;
p %bar;
Output:
{
A {
qt "a",
st "a"
},
B {
qt "b",
st "b"
}
}
{
A {
qt "a",
st "a"
},
B {
qt "b",
st "b"
}
}

What does it mean to have a perl hash{}{}

My professor has some syntax on a slide that I do not understand.
In perl there is:
$hash{$string}{$anotherString}++;
What does this syntax mean? If it were:
$hash{$string}{$int}++;
Would it be increment the value?
When I print using
while( my( $key, $value ) = each %hash ){print "$key: $value\n";}
My output is
"key": HASH(0xbe0200)
That is a two-dimensional hash, a hash of hashes. It is easy to keep track of structures in Perl once you realize that any single value is in fact a scalar. In the case of multidimensional structures, the scalar value is a reference. For example:
my %outer = ( "foo" => { "bar" => 1 } );
The inner part { "bar" => 1 } is a hash reference. The use of { } in assignment denotes an anonymous hash. This is similar to:
my %inner = ( "bar" => 1 );
my %outer = ( "foo" => \%inner );
Now when you want to reference a value in %inner, you use the first key to access the hash reference, and the second key to access the value in %inner:
print $outer{"foo"}{"bar"}; # prints 1
And when you use the increment operator ++ on a value, it is incremented:
$outer{"foo"}{"bar"}++; # the value is now 2
$hash{string1}{string2}
is a shorter equivalent of
$hash{string1}->{string2}
i.e. it returns a value from a hash of hashes.
By applying the ++ operator, the value in the inner hash is incremented.
My output is "key": HASH(0xbe0200)
That strange output means that what you are trying to print is actually a hash reference:
use strict;
use warnings;
use 5.016; #allows you to use say(), which is equivalent to print()
#plus a newline at the end
my $href = {
a => 1,
b => 2,
};
say $href;
--output:--
HASH(0x100826698)
Or,
my %hash = (
a => 1,
b => 2,
);
say \%hash;
--output:--
HASH(0x1008270a0)
The \ operator gets the reference for the thing on its right hand side.
The easiest way to print the actual hash is using Data::Dumper, which is something you can and will use all the time:
use strict;
use warnings;
use 5.016;
use Data::Dumper;
my $href = {
a => 1,
b => 2,
};
say Dumper($href);
$VAR1 = {
'a' => 1,
'b' => 2
};
Like use warnings;, I consider use Data::Dumper; mandatory for every program.
So, when you see strange output, like HASH(0xbe0200), use Data::Dumper on the value:
my %hash = (
a => 1,
b => { hello => 2, goodbye => 3},
);
while( my( $key, $value ) = each %hash ){
say $key;
say Dumper($value);
say '-' x 10;
}
--output:--
a
$VAR1 = 1;
----------
b
$VAR1 = {
'hello' => 2,
'goodbye' => 3
};
----------
Or, alternatively just use Data::Dumper on the whole structure:
my %hash = (
a => 1,
b => { hello => 2, goodbye => 3},
);
say Dumper(\%hash);
--output:--
$VAR1 = {
'a' => 1,
'b' => {
'hello' => 2,
'goodbye' => 3
}
};
Note that Dumper() is used to show the contents of a hash reference(or any other reference), so if your variable is not a reference, e.g. %hash, then you must turn it into a reference using the \ operator, e.g. \%hash.
Now, if you have this hash:
my %hash = (
a => 1,
b => { hello => 2, goodbye => 3},
);
...to retrieve the value corresponding to 'goodbye', you can write:
say $hash{b}{goodbye}; #=>3
$hash{b} returns the hash (reference) { hello => 2, goodbye => 3}, and you can retrieve values from that hash by using the subscripts {hello} or {goodbye}.
Alternatively, you can write this:
my %hash = (
a => 1,
b => { hello => 2, goodbye => 3},
);
my $string = 'b';
my $anotherString = 'goodbye';
say $hash{$string}{$anotherString}; #=>3
And to increment the value 3 in the hash, you can write:
my $result = $hash{$string}{$anotherString}++;
say $result; #=>3
say $hash{$string}{$anotherString}; #=>4
The postfix ++ operator actually increments the value after the current operation, so $result is 3, then the value in the hash is incremented to 4, something like this:
my $temp = $hash{$string}{$anotherString};
$hash{$string}{$anotherString} = $hash{$string}{$anotherString} + 1;
my $result = $temp;
If you want the increment to happen before the current operation, then you can use the prefix ++ operator:
my $result = ++$hash{$string}{$anotherString};
say $result; #=>4
say $hash{$string}{$anotherString}; #=>4
Finally, if the value at $hash{$string}{$anotherString} is not a number, e.g. 'green', you will get something strange:
my %hash = (
a => 1,
b => { hello => 2, goodbye => 'green'},
);
my $string = 'b';
my $anotherString = 'goodbye';
my $result = $hash{$string}{$anotherString}++;
say $hash{$string}{$anotherString};
--output:--
greeo
perl has a notion that the string that comes after the string 'green' is the string 'greeo' because the letter 'o' comes after the letter 'n' in the alphabet. And if the string you incremented were 'greez' the output would be:
greez original
grefa output
The next letter after 'z' is to start over with 'a', but just like when you increment 9 by 1 and get 10, the increment for 'z' carries over to the column on the left, incrementing that letter by 1, producing the 'f'. Ha!

Can we use Storable's store method on a tied hash?

Tried to store a hash reference in a Storable file and it worked fine. However I also had a requirement to keep the keys in a sorted order - so I used the following
tie %$hashref, 'Tie::IxHash';
store $hashref, $filename;
But this doesn't work - the file gets created but its only 50 bytes in size and when I use retrieve(), I get an empty hash.
I tried using Tie::IxHash::Easy (because my hash is a hash of hashes) but to no avail. Any help?
You need to tie the hashref before you populate it
use strict; use warnings;
use Storable;
use Tie::IxHash;
use Data::Dumper;
my $filename= 'xxx';
my $hashref;
tie %{$hashref}, 'Tie::IxHash';
$hashref->{b}={c=>2, d=>{e=>3}};
$hashref->{a}=1;
print Dumper(before=>$hashref);
store $hashref, $filename;
print Dumper(after=>retrieve('xxx'));
returns
$VAR1 = 'before';
$VAR2 = {
'b' => {
'c' => 2,
'd' => {
'e' => 3
}
},
'a' => 1
};
$VAR1 = 'after';
$VAR2 = {
'b' => {
'c' => 2,
'd' => {
'e' => 3
}
},
'a' => 1
};

Adding hash keys

I am adding data to a hash using an incrementing numeric key starting at 0. The key/value is fine. When I add the second one, the first key/value pair points back to the second. Each addition after that replaces the value of the second key and then points back to it. The Dumper output would be something like this.
$VAR1 = { '0' => { ... } };
After the first key/value is added. After the second one is added I get
$VAR1= { '1' => { ... }, '0' => $VAR1->{'1} };
After the third key/value is added, it looks like this.
$VAR1 = { '1' => { ... }, '0' => $VAR1->{'1'}, '2' => $VAR1->{'1'} };
My question is why is it doing this? I want each key/value to show up in the hash. When I iterate through the hash I get the same data for every key/value. How do I get rid of the reference pointers to the second added key?
You are setting the value of every element to a reference to the same hash. Data::Dumper is merely reflecting that.
If you're using Data::Dumper as a serializing tool (yuck!), then you should set $Data::Dumper::Purity to 1 to get something eval can process.
use Data::Dumper qw( Dumper );
my %h2 = (a=>5,b=>6,c=>7);
my %h;
$h{0} = \%h2;
$h{1} = \%h2;
$h{2} = \%h2;
print("$h{0}{c} $h{2}{c}\n");
$h{0}{c} = 9;
print("$h{0}{c} $h{2}{c}\n");
{
local $Data::Dumper::Purity = 1;
print(Dumper(\%h));
}
Output:
7 7
9 9
$VAR1 = {
'0' => {
'c' => 9,
'a' => 5,
'b' => 6
},
'1' => {},
'2' => {}
};
$VAR1->{'0'} = $VAR1->{'1'};
$VAR1->{'2'} = $VAR1->{'1'};
If, on the other hand, you didn't mean to use store references to different hashes, you could use
# Shallow copies
$h{0} = { %h2 }; # { ... } means do { my %anon = ( ... ); \%anon }
$h{1} = { %h2 };
$h{2} = { %h2 };
or
# Deep copies
use Storable qw( dclone );
$h{0} = dclone(\%h2);
$h{1} = dclone(\%h2);
$h{2} = dclone(\%h2);
Output:
7 7
9 7
$VAR1 = {
'0' => {
'a' => 5,
'b' => 6,
'c' => 9
},
'1' => {
'a' => 5,
'b' => 6,
'c' => 7
},
'2' => {
'a' => 5,
'b' => 6,
'c' => 7
}
};
You haven't posted the actual code you're using to build the hash, but I assume it looks something like this:
foreach my $i (1 .. 3) {
%hash2 = (number => $i, foo => "bar", baz => "whatever");
$hash1{$i} = \%hash2;
}
(Actually, I'll guess that, in your actual code, you're probably reading data from a file in a while (<>) loop and assigning values to %hash2 based on it, but the foreach loop will do for demonstration purposes.)
If you run the code above and dump the resulting %hash1 using Data::Dumper, you'll get the output:
$VAR1 = {
'1' => {
'baz' => 'whatever',
'number' => 3,
'foo' => 'bar'
},
'3' => $VAR1->{'1'},
'2' => $VAR1->{'1'}
};
Why does it look like that? Well, it's because the values in %hash1 are all references pointing to the same hash, namely %hash2. When you assign new values to %hash2 in your loop, those values will overwrite the old values in %hash2, but it will still be the same hash. Data::Dumper is just highlighting that fact.
So, how can you fix it? Well, there are (at least) two ways. One way is to replace \%hash2, which gives a reference to %hash2, with { %hash2 }, which copies the contents of %hash2 into a new anonymous hash and returns a reference to that:
foreach my $i (1 .. 3) {
%hash2 = (number => $i, foo => "bar", baz => "whatever");
$hash1{$i} = { %hash2 };
}
The other (IMO preferable) way is to declare %hash2 as a (lexically scoped) local variable within the loop using my:
foreach my $i (1 .. 3) {
my %hash2 = (number => $i, foo => "bar", baz => "whatever");
$hash1{$i} = \%hash2;
}
This way, each iteration of the loop will create a new, different hash named %hash2, while the hashes created on previous iterations will continue to exist (since they're referenced from %hash1) independently.
By the way, you wouldn't have had this problem in the first place if you'd followed standard Perl best practices, specifically:
Always use strict; (and use warnings;). This would've forced you to declare %hash2 with my (although it wouldn't have forced you to do so inside the loop).
Always declare local variables in the smallest possible scope. In this case, since %hash2 is only used within the loop, you should've declared it inside the loop, like above.
Following these best practices, the example code above would look like this:
use strict;
use warnings;
use Data::Dumper qw(Dumper);
my %hash1;
foreach my $i (1 .. 3) {
my %hash2 = (number => $i, foo => "bar", baz => "whatever");
$hash1{$i} = \%hash2;
}
print Dumper(\%hash1);
which, as expected, will print:
$VAR1 = {
'1' => {
'baz' => 'whatever',
'number' => 1,
'foo' => 'bar'
},
'3' => {
'baz' => 'whatever',
'number' => 3,
'foo' => 'bar'
},
'2' => {
'baz' => 'whatever',
'number' => 2,
'foo' => 'bar'
}
};
It's hard to see what the problem is when you don't post the code or the actual results of Data::Dumper.
There is one thing you should know about Data::Dumper: When you dump an array or (especially) a hash, you should dump a reference to it. Otherwise, Data::Dumper will treat it like a series of variables. Also notice that hashes do not remain in the order you create them. I've enclosed an example below. Make sure that your issue isn't related to a confusing Data::Dumper output.
Another question: If you're keying your hash by sequential keys, would you be better off with an array?
If you can, please edit your question to post your code and the ACTUAL results.
use strict;
use warnings;
use autodie;
use feature qw(say);
use Data::Dumper;
my #array = qw(one two three four five);
my %hash = (one => 1, two => 2, three => 3, four => 4);
say "Dumped Array: " . Dumper #array;
say "Dumped Hash: " . Dumper %hash;
say "Dumped Array Reference: " . Dumper \#array;
say "Dumped Hash Reference: " . Dumper \%hash;
The output:
Dumped Array: $VAR1 = 'one';
$VAR2 = 'two';
$VAR3 = 'three';
$VAR4 = 'four';
$VAR5 = 'five';
Dumped Hash: $VAR1 = 'three';
$VAR2 = 3;
$VAR3 = 'one';
$VAR4 = 1;
$VAR5 = 'two';
$VAR6 = 2;
$VAR7 = 'four';
$VAR8 = 4;
Dumped Array Reference: $VAR1 = [
'one',
'two',
'three',
'four',
'five'
];
Dumped Hash Reference: $VAR1 = {
'three' => 3,
'one' => 1,
'two' => 2,
'four' => 4
};
The reason it is doing this is you are giving it the same reference to the same hash.
Presumably in a loop construct.
Here is a simple program which has this behaviour.
use strict;
use warnings;
# always use the above two lines until you
# understand completely why they are recommended
use Data::Printer;
my %hash;
my %inner; # <-- wrong place to put it
for my $index (0..5){
$inner{int rand} = $index; # <- doesn't matter
$hash{$index} = \%inner;
}
p %hash;
To fix it just make sure that you are creating a fresh hash reference every time through the loop.
use strict;
use warnings;
use Data::Printer;
my %hash;
for my $index (0..5){
my %inner; # <-- place the declaration here instead
$inner{int rand} = $index; # <- doesn't matter
$hash{$index} = \%inner;
}
p %hash;
If you are only going to use numbers for your indexes, and they are monotonically increasing starting from 0, then I would recommend using an array.
An array would be faster and more memory efficient.
use strict;
use warnings;
use Data::Printer;
my #array; # <--
for my $index (0..5){
my %inner;
$inner{int rand} = $index;
$array[$index] = \%inner; # <--
}
p #array;

How to shuffle the values in a hash?

I have a hash of string IDs. What is the best way to shuffle the IDs?
As an example, my hash assigns the following IDs:
this => 0
is => 1
a => 2
test => 3
Now I'd like to randomly shuffle that. An example outcome would be:
this => 1
is => 0
a => 3
test => 2
You could use the shuffle method in List::Util to help out:
use List::Util qw(shuffle);
...
my #values = shuffle(values %hash);
map { $hash{$_} = shift(#values) } (keys %hash);
A hash slice would be the clearest way to me:
#!/usr/bin/perl
use strict;
use warnings;
use List::Util qw/shuffle/;
use Data::Dumper;
my %h = (
this => 0,
is => 1,
a => 2,
test => 3,
);
#h{keys %h} = shuffle values %h;
print Dumper \%h;
This has a drawback in that huge hashes would take up a lot of memory as you pull all of their keys and values out. A more efficient (from a memory standpoint) solution would be:
#!/usr/bin/perl
use strict;
use warnings;
use List::Util qw/shuffle/;
use Data::Dumper;
my %h = (
this => 0,
is => 1,
a => 2,
test => 3,
);
{ #bareblock to cause #keys to be garbage collected
my #keys = shuffle keys %h;
while (my $k1 = each %h) {
my $k2 = shift #keys;
#h{$k1, $k2} = #h{$k2, $k1};
}
}
print Dumper \%h;
This code has the benefit of only having to duplicate the keys (rather than the keys and values).
The following code doesn't randomize the values (except on Perl 5.8.1 where the order of keys is guaranteed to be random), but it does mix up the order. It does have the benefit of working in place without too much extra memory usage:
#!/usr/bin/perl
use strict;
use warnings;
use List::Util qw/shuffle/;
use Data::Dumper;
my %h = (
this => 0,
is => 1,
a => 2,
test => 3,
);
my $k1 = each %h;
while (defined(my $k2 = each %h)) {
#h{$k1, $k2} = #h{$k2, $k1};
last unless defined($k1 = each %h);
}
print Dumper \%h;