I have a Hash of following structure in perl -
my %testHash = (
KeyL1 => {
KeyLL1 => {
KeyLLL1 => [1,2],
KeyLLL2 => [2,3],
},
KeyLL2 => {
KeyLLL1 => [1,2],
KeyLLL2 => [2,3],
},
KeyLL3 => {
KeyLLL1 => [1,2],
KeyLLL2 => [2,3],
},
},
KeyL2 => {
KeyLL1 => {
KeyLLL1 => [1,2],
KeyLLL2 => [2,3],
},
KeyLL2 => {
KeyLLL1 => [1,2],
KeyLLL2 => [2,3],
},
KeyLL3 => {
KeyLLL1 => [1,2],
KeyLLL2 => [2,3],
},
},
);
Now, when I am trying to access it the following way, I am getting 'undef' as a result
my %tempHash = $testHash{'KeyL1'};
print Data::Dumper::Dumper($tempHash{'KeyLL1'});
print Data::Dumper::Dumper($tempHash{'KeyLL1'}{'KeyLLL1'});
Result --
$VAR1 = undef; $VAR1 = undef;
Please point to me what am I doing wrong. I am pretty new to perl.
The value of $testHash{'KeyL1'} is a hashref, not a hash.
Hashrefs are scalars. my %tempHash = is not expecting a scalar.
You need to dereference it:
my %tempHash = %{$testHash{'KeyL1'}};
Also, you could do it this way if its just about viewing the structures.
Also try:
print Dumper $testHash{KeyL1} ;
print Dumper $testHash{KeyL1}{KeyLL1} ;
print Dumper $testHash{KeyL1}{KeyLL1}{KeyLLL1} ;
Output:
%_Host#User> ./hash.pl
$VAR1 = {
'KeyLL1' => {
'KeyLLL2' => [
2,
3
],
'KeyLLL1' => [
1,
2
]
},
'KeyLL2' => {
'KeyLLL2' => [
2,
3
],
'KeyLLL1' => [
1,
2
]
},
'KeyLL3' => {
'KeyLLL2' => [
2,
3
],
'KeyLLL1' => [
1,
2
]
}
};
$VAR1 = {
'KeyLLL2' => [
2,
3
],
'KeyLLL1' => [
1,
2
]
};
$VAR1 = [
1,
2
];
%_Host#User>
I am using Amazon's AWS command-line tool to extract all instances that we have running. This produces a .json file with (modified) the following format:
{
"Reservations": [
{
"OwnerId": "8172695814",
"ReservationId": "q-9d77c34a",
"Groups": [],
"Instances": [
{
...
"LaunchTime": "2014-08-14T11:37:29.000Z",
"Tags": [
{
"Value": "Server 5",
"Key": "Name"
}
],
"ProductCodes": [],
...
I want to use this output as the source for a dashboard-like application, built in Perl. My problem is that the server list in the .json file is not sorted the way I want: I would like to see them ordered by Tag name (the values behind Tags-Value). So in the example above, any Server 1 to 4 should be above the Server 5, and all others below.
(edited) I can loop throught the values in order to show them in the dashboard like this:
foreach my $instance_list (#instances) {
foreach my $instance (#$instance_list){
if ( $$instance{'Instances'}[0]{'State'}{'Name'} !~ m/terminated/io ) {
if ( $$instance{'Instances'}[0]{'Tags'}[0]{'Value'} =~ m/Server.*/o ) {
...
But now the challenge is to get the elements sorted in the proer order. Can anyone tell me on how this can be done in Perl? Thanks!
You can use custom sort method with perl : #sorted = sort { $a cmp $b } #list;
If you play this example:
use strict;
use Data::Dumper;
my #servers = (
{ name => "server1", launchtime => "2014-08-14T11:37:29.000Z" },
{ name => "server2", launchtime => "2014-08-15T11:37:29.000Z" },
{ name => "server5", launchtime => "2014-08-16T11:37:29.000Z" },
{ name => "server4", launchtime => "2014-08-17T11:37:29.000Z" },
{ name => "server3", launchtime => "2014-08-12T11:37:29.000Z" },
);
print Dumper( \#servers);
my #default_sorted = sort #servers;
my #custom_sorted = sort {$a->{name} cmp $b->{name}} #servers;
The result will be:
default_sorted => server5, 4, 3, 1, 2 (because of the lauchtime
probaly)
custom_sorted => server1, 2, 3, 4, 5
Something like this seems to do what you want.
#!/usr/bin/perl
use strict;
use warnings;
use 5.010;
my $servers = {
Reservations => [ {
Something => 'foo',
Instances => [ {
Tags => [ {
Value => 1,
Key => 'Name',
} ],
} ],
}, {
Something => 'bar',
Instances => [ {
Tags => [ {
Value => 0,
Key => 'Name',
} ],
} ],
} ],
};
foreach (sort { $a->{Instances}[0]{Tags}[0]{Value} <=>
$b->{Instances}[0]{Tags}[0]{Value} }
#{$servers->{Reservations}}) {
say $_->{Something};
}
Prints 'bar' followed by 'foo' - which is the correct order.
Finally got it working. The probles was that the Reservations element was there double, so I needed to fetch the first element to work with. This is what it is like now:
#{$instances[0]} = sort { $a->{'Instances'}[0]{'Tags'}[0]{'Value'} cmp $b->{'Instances'}[0]{'Tags'}[0]{'Value'} } #{$instances_unsorted[0]};
Thank for the tips!
When I do
#!/usr/bin/perl -w
use strict;
use Net::LDAP;
use Data::Dumper;
my $dn="...";
my $password="xxx";
my $ldap = Net::LDAP->new('example.com') or die "$#";
my $mesg = $ldap->bind($dn, password => $password);
if ($mesg->code) { die "uuuu $mesg"; }
$mesg = $ldap->search(
base => "dc=example,dc=com",
filter => "(name=LIST)",
);
print Dumper $mesg;
I get
$VAR1 = bless( {
'parent' => bless( {
...
}, 'Net::LDAP' ),
'entries' => [
bless( {
'changes' => [],
'changetype' => 'modify',
'asn' => {
'objectName' => 'CN=LIST,OU=test group,OU=M,OU=I,DC=example,DC=com',
'attributes' => [
{
'type' => 'objectClass',
'vals' => [
'top',
'group'
]
},
{
'type' => 'cn',
'vals' => [
'LIST'
]
},
{
'type' => 'member',
'vals' => [
'CN=user1,OU=BaseUsers,DC=example,DC=com',
'CN=user2,OU=BaseUsers,DC=example,DC=com',
]
},
...
where I would only like to output those from member that have in their object
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=example,DC=com
Does anyone know how to do that?
foreach my $entry (#{$mesg->{'entries'}})
{
my $match = 0;
my $name = $entry->{'asn'}->{'objectName'};
foreach my $attr (#{$entry->{'asn'}->{'attributes'}})
{
if('member' eq $attr->{'type'})
{
foreach my $val (#{$attr->{'vals'}})
{
if($val =~ /^CN=.*,CN=.*,CN=.*,DC=example,DC=com$/)
{
$match = 1;
last;
}
}
}
}
if($match)
{
print $name;
}
}
For your example data above this will return no matches since none of the "members" match the search pattern you specified. Also I wasn't sure if you wanted to output the object name (as per my code) or the matching string. If the latter you don't need $match, simply put a print in the innermost block.
In Perl, how do I get this:
$VAR1 = { '999' => { '998' => [ '908', '906', '0', '998', '907' ] } };
$VAR1 = { '999' => { '991' => [ '913', '920', '918', '998', '916', '919', '917', '915', '912', '914' ] } };
$VAR1 = { '999' => { '996' => [] } };
$VAR1 = { '999' => { '995' => [] } };
$VAR1 = { '999' => { '994' => [] } };
$VAR1 = { '999' => { '993' => [] } };
$VAR1 = { '999' => { '997' => [ '986', '987', '990', '984', '989', '988' ] } };
$VAR1 = { '995' => { '101' => [] } };
$VAR1 = { '995' => { '102' => [] } };
$VAR1 = { '995' => { '103' => [] } };
$VAR1 = { '995' => { '104' => [] } };
$VAR1 = { '995' => { '105' => [] } };
$VAR1 = { '995' => { '106' => [] } };
$VAR1 = { '995' => { '107' => [] } };
$VAR1 = { '994' => { '910' => [] } };
$VAR1 = { '993' => { '909' => [] } };
$VAR1 = { '993' => { '904' => [] } };
$VAR1 = { '994' => { '985' => [] } };
$VAR1 = { '994' => { '983' => [] } };
$VAR1 = { '993' => { '902' => [] } };
$VAR1 = { '999' => { '992' => [ '905' ] } };
to this:
$VAR1 = { '999:' => [
{ '992' => [ '905' ] },
{ '993' => [
{ '909' => [] },
{ '904' => [] },
{ '902' => [] }
] },
{ '994' => [
{ '910' => [] },
{ '985' => [] },
{ '983' => [] }
] },
{ '995' => [
{ '101' => [] },
{ '102' => [] },
{ '103' => [] },
{ '104' => [] },
{ '105' => [] },
{ '106' => [] },
{ '107' => [] }
] },
{ '996' => [] },
{ '997' => [ '986', '987', '990', '984', '989', '988' ] },
{ '998' => [ '908', '906', '0', '998', '907' ] },
{ '991' => [ '913', '920', '918', '998', '916', '919', '917', '915', '912', '914' ] }
]};
I think this is closer than anybody else has gotten:
This does most of what you want. I did not store things in arrays of singular
hashes, as I don't feel that that is useful.
Your scenario is not a regular one. I've tried to genericize this to some extent,
but was not possible to overcome the singularity of this code.
First of all because it appears you want to collapse everything with the same
id into a merged entity (with exceptions), you have to descend through the structure
pulling the definitions of the entities. Keeping track of levels, because you
want them in the form of a tree.
Next, you assemble the ID table, merging entities as possible. Note that you
had 995 defined as an empty array one place and as a level another. So given
your output, I wanted to overwrite the empty list with the hash.
After that, we need to move the root to the result structure, descending that in order
to assign canonical entities to the identifiers at each level.
Like I said, it's not anything that regular. Of course, if you still want a list
of hashes which are no more than pairs, that's an exercise left to you.
use strict;
use warnings;
# subroutine to identify all elements
sub descend_identify {
my ( $level, $hash_ref ) = #_;
# return an expanding list that gets populated as we desecend
return map {
my $item = $hash_ref->{$_};
$_ => ( $level, $item )
, ( ref( $item ) eq 'HASH' ? descend_identify( $level + 1, $item )
: ()
)
;
} keys %$hash_ref
;
}
# subroutine to refit all nested elements
sub descend_restore {
my ( $hash, $ident_hash ) = #_;
my #keys = keys %$hash;
#$hash{ #keys } = #$ident_hash{ #keys };
foreach my $h ( grep { ref() eq 'HASH' } values %$hash ) {
descend_restore( $h, $ident_hash );
}
return;
}
# merge hashes, descending down the hash structures.
sub merge_hashes {
my ( $dest_hash, $src_hash ) = #_;
foreach my $key ( keys %$src_hash ) {
if ( exists $dest_hash->{$key} ) {
my $ref = $dest_hash->{$key};
my $typ = ref( $ref );
if ( $typ eq 'HASH' ) {
merge_hashes( $ref, $src_hash->{$key} );
}
else {
push #$ref, $src_hash->{$key};
}
}
else {
$dest_hash->{$key} = $src_hash->{$key};
}
}
return;
}
my ( %levels, %ident_map, %result );
#descend through every level of hash in the list
# #hash_list is assumed to be whatever you Dumper-ed.
my #pairs = map { descend_identify( 0, $_ ); } #hash_list;
while ( #pairs ) {
my ( $key, $level, $ref ) = splice( #pairs, 0, 3 );
$levels{$key} |= $level;
# if we already have an identity for this key, merge the two
if ( exists $ident_map{$key} ) {
my $oref = $ident_map{$key};
my $otyp = ref( $oref );
if ( $otyp ne ref( $ref )) {
# empty arrays can be overwritten by hashrefs -- per 995
if ( $otyp eq 'ARRAY' && #$oref == 0 && ref( $ref ) eq 'HASH' ) {
$ident_map{$key} = $ref;
}
else {
die "Uncertain merge for '$key'!";
}
}
elsif ( $otyp eq 'HASH' ) {
merge_hashes( $oref, $ref );
}
else {
#$oref = sort { $a <=> $b || $a cmp $b } keys %{{ #$ref, #$oref }};
}
}
else {
$ident_map{$key} = $ref;
}
}
# Copy only the keys that do not appear at higher levels to the
# result hash
if ( my #keys = grep { !$levels{$_} } keys %ident_map ) {
#result{ #keys } = #ident_map{ #keys } if #keys;
}
# then step through the hash to make sure that the entries at
# all levels are equal to the identity
descend_restore( \%result, \%ident_map );
Use CPAN! Try Hash::Merge
# OO interface.
my $merge = Hash::Merge->new( 'LEFT_PRECEDENT' );
my %c = %{ $merge->merge( \%a, \%b ) };
See CPAN for more info, it pretty much does everything you would want to, and is fully customizable.
Give this recursive solution a try:
# XXX: doesn't handle circular problems...
sub deepmerge {
my (#structs) = #_;
my $new;
# filter out non-existant structs
#structs = grep {defined($_)} #structs;
my $ref = ref($structs[0]);
if (not all(map {ref($_) eq $ref} #structs)) {
warn("deepmerge: all structs are not $ref\n");
}
my #tomerge = grep {ref($_) eq $ref} #structs;
return qr/$tomerge[0]/ if scalar(#tomerge) == 1 and $ref eq 'Regexp';
return $tomerge[0] if scalar(#tomerge) == 1;
if ($ref eq '') {
$new = pop(#tomerge); # prefer farthest right
}
elsif ($ref eq 'Regexp') {
$new = qr/$tomerge[$#tomerge]/;
}
elsif ($ref eq 'ARRAY') {
$new = [];
for my $i (0 .. max(map {scalar(#$_) - 1} #tomerge)) {
$new->[$i] = deepmerge(map {$_->[$i]} #tomerge);
}
}
elsif ($ref eq 'HASH') {
$new = {};
for my $key (uniq(map {keys %$_} #tomerge)) {
$new->{$key} = deepmerge(map {$_->{$key}} #tomerge);
}
}
else {
# ignore all other structures...
$new = '';
}
return $new;
}
Modify it to your hearts content to achieve the desired result.
Upon further investigation, I noticed you're merging them in some different way than the above algorithm. Maybe just use this as an example then. Mine does this:
deepmerge({k => 'v'}, {k2 => 'v2'});
# returns {k => 'v', k2 => 'v2'}
And similar things for arrays.
I indented your wanted output as it was hard to read, for the benefit of other people who want to answer. I'm still thinking of an answer.
$VAR1 = { '999:' => [
{ '992' => [ '905' ] },
{ '993' => [
{ '909' => [] },
{ '904' => [] },
{ '902' => [] }
]
},
{ '994' => [
{ '910' => [] },
{ '985' => [] },
{ '983' => [] }
]
},
{ '995' => [
{ '101' => [] },
{ '102' => [] },
{ '103' => [] },
{ '104' => [] },
{ '105' => [] },
{ '106' => [] },
{ '107' => [] }
]
},
{ '996' => [] },
{ '997' => [ '986', '987', '990', '984', '989', '988' ] },
{ '998' => [ '908', '906', '0', '998', '907' ] },
{ '991' => [ '913', '920', '918', '998', '916', '919', '917', '915', '912', '914' ] }
]
};
I don't see the point of all those single entry hashes though, would not the following be better?
$VAR1 = { '999:' => {
'992' => [ '905' ],
'993' => {
'909' => [],
'904' => [],
'902' => []
},
'994' => {
'910' => [],
'985' => [],
'983' => []
},
'995' => {
'101' => [],
'102' => [],
'103' => [],
'104' => [],
'105' => [],
'106' => [],
'107' => []
},
'996' => [],
'997' => [ '986', '987', '990', '984', '989', '988' ],
'998' => [ '908', '906', '0', '998', '907' ],
'991' => [ '913', '920', '918', '998', '916', '919', '917', '915', '912', '914' ]
}
};
Assuming the above data is in a file dump.txt, you can eval it piece by piece.
Updated code below
use strict;
use File::Slurp;
my $final_data = {};
my #data = map {eval $_} (read_file("dump.txt") =~ /\$VAR1 = ([^;]+);/gs);
foreach my $element (#data) {
my $key = (keys %$element)[0];
$final_data->{$key} ||= [];
push #{$final_data->{$key}}, $element->{$key}
};
use Data::Dumper;
print Data::Dumper->Dump([$final_data]);
If you want to completely deep merge, you can at the end pass $final_data through this (not tested!!!) deep merger:
# Merge an array of hashes as follows:
# IN: [ { 1 => 11 }, { 1 => 12 },{ 2 => 22 } ]
# OUT: { 1 => [ 11, 12 ], 2 => [ 22 ] }
# This is recursive - if array [11,12] was an array of hashrefs, we merge those too
sub merge_hashes {
my $hashes = #_[0];
return $hashes unless ref $hashes eq ref []; # Hat tip to brian d foy
return $hashes unless grep { ref #_ eq ref {} } #$hashes; # Only merge array of hashes
my $final_hashref = {};
foreach my $element (#$hashes) {
foreach my $key (keys %$element) {
$final_hashref->{$key} ||= [];
push #{ $final_hashref->{$key} }, $element->{$key};
}
}
foreach my $key (keys %$final_hashref) {
$final_hashref->{$key} = merge_hashes($final_hashref->{$key});
}
return $final_hashref;
}
Use push and autovivification.
Start with the usual front matter:
#! /usr/bin/perl
use warnings;
use strict;
Read your sample input from the DATA filehandle and create a datastructure similar to the one you dumped:
my #hashes;
while (<DATA>) {
my $VAR1;
$VAR1 = eval $_;
die $# if $#;
push #hashes => $VAR1;
}
Your input has two cases:
A reference to an array that contains data to be merged with its cousins that have the same "key path."
Otherwise, it's a reference to a hash that contains a reference to an array from case 1 at some depth, so we strip off the outermost layer and keep digging.
Note the use of $_[0]. The semantics of Perl subroutines are such that the values in #_ are aliases rather than copies. This lets us call merge directly without having to first create a bunch of scaffolding to hold the merged contents. The code will break if you copy the value instead.
sub merge {
my $data = shift;
if (ref($data) eq "ARRAY") {
push #{ $_[0] } => #$data;
}
else {
foreach my $k (%$data) {
merge($data->{$k} => $_[0]{$k});
}
}
}
Now we walk #hashes and incrementally merge their contents into %merged.
my %merged;
foreach my $h (#hashes) {
foreach my $k (keys %$h) {
merge $h->{$k} => $merged{$k};
}
}
We don't know in what order the values arrived, so run a final cleanup pass to sort the arrays:
sub sort_arrays {
my($root) = #_;
if (ref($root) eq "ARRAY") {
#$root = sort { $a <=> $b } #$root;
}
else {
sort_arrays($root->{$_}) for keys %$root;
}
}
sort_arrays \%merged;
The Data::Dumper module is great for quick debugging!
use Data::Dumper;
$Data::Dumper::Indent = 1;
print Dumper \%merged;
Place a copy of the input from your question into the special DATA filehandle:
__DATA__
$VAR1 = { '999' => { '998' => [ '908', '906', '0', '998', '907' ] } };
$VAR1 = { '999' => { '991' => [ '913', '920', '918', '998', '916', '919', '917', '915', '912', '914' ] } };
$VAR1 = { '999' => { '996' => [] } };
$VAR1 = { '999' => { '995' => [] } };
$VAR1 = { '999' => { '994' => [] } };
$VAR1 = { '999' => { '993' => [] } };
$VAR1 = { '999' => { '997' => [ '986', '987', '990', '984', '989', '988' ] } };
$VAR1 = { '995' => { '101' => [] } };
$VAR1 = { '995' => { '102' => [] } };
$VAR1 = { '995' => { '103' => [] } };
$VAR1 = { '995' => { '104' => [] } };
$VAR1 = { '995' => { '105' => [] } };
$VAR1 = { '995' => { '106' => [] } };
$VAR1 = { '995' => { '107' => [] } };
$VAR1 = { '994' => { '910' => [] } };
$VAR1 = { '993' => { '909' => [] } };
$VAR1 = { '993' => { '904' => [] } };
$VAR1 = { '994' => { '985' => [] } };
$VAR1 = { '994' => { '983' => [] } };
$VAR1 = { '993' => { '902' => [] } };
$VAR1 = { '999' => { '992' => [ '905' ] } };
A sample of the output is below:
'994' => {
'910' => [],
'985' => [],
'983' => []
},
'999' => {
'993' => [],
'992' => [
'905'
],
'997' => [
'984',
'986',
'987',
'988',
'989',
'990'
],
wow. thanks so much everyone (especially Axeman)! sorry for the lack of code or clarification, I was trying to generate a tree, and did try Hash::Merge, but could not for the life of me resolve the coined-995 problem of replacing the empty 995 with the non-empty 995; Axeman's solution works beautifully and I really appreciate the help/collaboration! (also tried the others and it either did the same thing as Hash::Merge, or it actually got rid of some branches).
some background on the input: had a set of hashes, each had keys (all same level) and two of which defined a) a parent to another, and b) itself (the rest were children), and so with a tree, i figured a hash was perfect, came up with a set of new hashes {a}->{b}->[c], and here we are...
again, thanks everyone and Axeman!