Goal: Remove a particular Value from an array
I have written a script and it works fine but I am not happy the way I have written it. So I am curious to know is there a better way to write it. please consider below use case:
I have a nested hash/hash/array...like below. I need to remove any array values which has local in their name:
#!/usr/bin/perl -w
use strict;
use Data::Dumper;
my $hash = { esx1 =>
{ cluster => "clu1",
fd => "fd1",
ds => [
'ds1',
'ds2',
'localds',
],
},
esx2 =>
{ cluster => "clu2",
fd => "fd2",
ds => [
'ds3',
'ds4',
'dslocal',
],
},
};
foreach my $a ( keys %$hash )
{
foreach ( 0..$#{ $hash->{$a}->{ds} } )
{
delete $hash->{$a}->{ds}->[$_] if $hash->{$a}->{ds}->[$_] =~ /local/i;
#{ $hash->{$a}->{ds} } = grep defined, #{ $hash->{$a}->{ds} };
}
}
print Dumper ($hash);
so the script deletes the "localds" and "dslocal" and keeps everything else intact.
Question:
Is there a cleaner way to write the foreach ( 0..$#{$hash->{$a}->{ds} } ) loop
If I do not write the grep line above, the resultant array has the value containing local deleted but is replaced by undef. Why is this happening.
Thanks.
delete is for elements in hashes. It happens to work on arrays by a quirk of implementation, but it shouldn't be relied on.
For arrays, you want to use splice.
splice #{ $ref->{to}->{array} }, $index, 1, ();
This replaces the 1-element sublist starting at $index with (), the empty list.
Why first iterate through array and delete elements and then look for "not-deleted" nodes (side note - this grep should be outside loop)? You can look for good nodes from very start! Replace entire loop with:
foreach my $a ( keys %$hash )
{
#{ $hash->{$a}->{ds} } = grep { !/local/i } #{ $hash->{$a}->{ds} };
}
for my $h (values %$hash){
$h->{ds} = [ grep { $_ !~ /local/i } #{$h->{ds}} ];
}
Not much neater but:
foreach my $a ( keys %$hash )
{
my $temp;
foreach ( #{ $hash->{$a}->{ds} } )
{
push(#$temp, $_) unless $_ =~ /local/i;
}
$hash->{$a}->{ds} = $temp;
}
Delete doesn't alter the array structure, it just alters the array content. Because of this in your method you need to grep for defined entries to create a new array of the structure you desire, then overwrite the old array.
edit:
This is better explained on perldoc page for delete
delete() may also be used on arrays and array slices, but its behavior is less straightforward. Although exists() will return false for deleted entries, deleting array elements never changes indices of existing values; use shift() or splice() for that. However, if all deleted elements fall at the end of an array, the array's size shrinks to the position of the highest element that still tests true for exists(), or to 0 if none do.
edit:
As pointed out by mob splice will do what you want:
foreach my $a ( keys %$hash )
{
for(0..$#{ $hash->{$a}->{ds} } )
{
splice( #{ $hash->{$a}->{ds} }, $_, 1 ) if $hash->{$a}->{ds}->[ $_ ] =~ /local/i;
}
}
You could use List::MoreUtils qw/first_idx/ to get the index of /local/ like so
first_idx { $_ =~ /local/i } #{$hash->{$a}->{ds}};
Then do what you want with that.
Related
I am trying to sort out defined parameters from a complex hash, using hash slices. Hash slices are great because they avoid lots of foreach and if defined so they simplify syntax greatly.
However, I'm clearly not doing this correctly:
use DDP;
my %hash = (
'a' => # first patient
{
'age' => 9,
'BMI' => 20
},
'b' =>
{
'age' => 8
}
);
my %defined_patients = grep {defined $hash{$_}{'age', 'BMI'}} keys %hash;
p %defined_patients;
The above code gives an empty hash, when I want it to return just patient a.
This question is similar to "no autovivication" pragma fails with grep in Perl and I've based my code on it.
I've also tried
my #defined_patients = grep {defined $hash{$_}{'age', 'BMI'}} keys %hash;
but that doesn't work either.
How can I use hash slices to grep patients with the defined keys?
If you want to check that none of the target keys are undefined, you have to check separately. This greps for undefined values:
grep { ! defined } #{ $hash{$_} }{ qw(age BMI) }
Notice that the hash slice
#{ $hash{$_} }{ qw(age BMI) }
In v5.24, you can use postfix dereferencing instead:
$hash{$_}->#{ qw(age BMI) }
But that grep has to fit in another one. Since you want the cases where all values are defined, you have to negate the result of the inner grep:
my #patients =
grep { ! grep { ! defined } $hash{$_}->#{ qw(age BMI) } }
keys %hash;
That's pretty ugly though. I'd probably do something simpler in a subroutine. This way you can handle any number of keys easily:
sub some_patients {
my( $hash, $keys ) = #_;
my #patient_keys;
foreach my $key ( keys %$hash ) {
next unless grep { ! defined } $hash{$key}->#{ #$keys };
push $key, #patient_keys;
}
return #patient_keys;
}
Now I simply call a subroutine instead of grokking multilevel greps:
my #patient_keys = some_patients( \%patients, [ qw(age BMI) ] );
Or, for something more targeted, maybe something like this that tests a particular sub-hash instead of the whole data structure:
sub has_defined_keys {
my( $hash, $keys ) = #_;
! grep { ! defined } $hash->#{#$keys}
}
my #target-keys = ...;
my #keys = grep {
has_defined_keys( $patients{$_}, \#target-keys )
} keys %patients;
Either way, when things start getting a bit too complex, use a subroutine to give those things names so you can hide the code in favor of something short.
I have a hash that looks like this:
my $hash = {
level1_f1 => {
level2_f1 => 'something',
level2_f2 => 'another thing'
},
level1_f2 => {
level2_f3 => 'yet another thing',
level2_f4 => 'bla bla'
level2_f5 => ''
}
...
}
I also got a list of values that correspond to the "level2" keys, which I want to know if thy exist in the hash.
#list = ("level2_f2", "level2_f4", "level2_f99")
I don't know which "level1" key each element of #list belongs to. The only way of finding if they existed I could think was using a foreach loop to go through #list, another foreach loop to go through the keys of %hash and checking
foreach my $i (#array) {
foreach my $k (keys %hash) {
if (exists $hash{$k}{$list[$i]})
}
}
but I wanted to know if there is a more eficient or maybe a more elegant way to do it. All the answers I found ask you to know the "level1" key, which I don't.
Thanks!!
Use values:
for my $inner_hash (values %$hash) {
say grep exists $inner_hash->{$_}, #list;
}
You have to loop all the level1 keys. But if you don't need to know which keys match and merely care for the existence of any, then you don't have to ask for each member of your list explicitly. You could say
foreach my $k (keys %hash) {
if ( #{ $hash{$k} }{ #list } )
{
}
}
The hash slice will return all values in the subhash which have matching keys in the list. Keys in the list that are not in the subhash get ignored.
Note however, that this does potentially more work than you may really need.
You don't need to iterate over "the entire hash".
You will necessarily over the elements of the outer hash since you want to check the value of each one, but you don't need to iterate over the elements of the inner hashes. Your solution already demonstrates that.
So your solution is as efficient as it can be, at least in terms of how well it scales. You can only perform small optimizations such as stopping as soon as a match is found.
for my $i (#list) {
while ( my (undef, $inner) = each(%hash) ) {
if (exists($inner->{$i}) {
...
last;
}
}
keys(%hash); # Reset iterator since it might not be exhausted.
}
As a micro optimization, it might be beneficial to invert the nesting of the loops.
my %list = map { $_ => 1 } #list;
while ( my (undef, $inner) = each(%hash) ) {
while (defined( my $k = each(%$inner) )) {
if ($list{$k}) {
delete($list{$k});
...
last if !keys(%list);
}
}
keys(%$inner); # Reset iterator since it might not be exhausted.
last if !keys(%list);
}
keys(%hash); # Reset iterator since it might not be exhausted.
If the hashes are small, these changes might actually slow things down.
Honestly, if there's truly a speed issue, the problem is that you used the wrong data structure for the type of query you want to run on it!
my $memType = [];
my $portOp = [];
my $fo = "aster.out.DRAMA.READ.gz";
if($fo =~/aster.out\.(.*)\.(.*)\.gz/){
push (#{$memType},$1);
push (#{$portOp},$2);
}
print Dumper #{$memType};
foreach my $mem (keys %{$portCapability->{#{$memType}}}){
//How to use the array ref memType inside a hash//
print "entered here\n";
//cannot post the rest of the code for obvious reasons//
}
I am not able to enter the foreach loop . Can anyone help me fix it?
Sorry this is not the complete code . Please help me.
%{$portCapability->{#{$memType}}}
This doesn't do what you may think it means.
You treat $portCapability->{#{$memType}} as a hash reference.
The #{$memType} is evaluated in scalar context, thus giving the size of the array.
I aren't quite sure what you want, but would
%{ $portCapability->{ $memType->[0] } }
work?
If, however, you want to slice the elements in $portCapability, you would need somethink like
#{ $portCapability }{ #$memType }
This evaluates to a list of hashrefs. You can then loop over the hashrefs, and loop over the keys in an inner loop:
for my $hash (#{ $portCapability }{ #$memType }) {
for my $key (keys %$hash) {
...;
}
}
If you want a flat list of all keys of the inner hashes, but don't need the hashes themselves, you could shorten above code to
for my $key (map {keys %$_} #{ $portCapability }{ #$memType }) {
...;
}
I think what you want is this:
my $foo = {
asdf => {
a => 1, b => 2,
},
foo => {
c => 3, d => 4
},
bar => {
e => 5, f => 6
}
};
my #keys = qw( asdf foo );
foreach my $k ( map { keys %{ $foo->{$_} } } #keys ) {
say $k;
}
But you do not know which of these $k belongs to which key of $foo now.
There's no direct way to get the keys of multiple things at the same time. It doesn't matter if these things are hashrefs that are stored within the same hashref under different keys, or if they are seperate variables. What you have to do is build that list yourself, by looking at each of the things in turn. That's simply done with above map statement.
First, look at all the keys in $foo. Then for each of these, return the keys inside that element.
my $memType = [];
my $portOp = [];
my $fo = “aster.out.DRAMA.READ.gz”;
if ($fo =~ /aster.out\.(\w+)\.(\w+)\.gz/ ) { #This regular expression is safer
push (#$memType, $1);
push (#$portOp, $2);
}
print Dumper “#$memType”; #should print “DRAMA”
#Now if you have earlier in your program the hash %portCapability, your code can be:
foreach $mem (#$memType) {
print $portCapability{$mem};
}
#or if you have the hash $portCapability = {…}, your code can be:
foreach $mem (#$memType) {
print $portCapability->{$mem};
}
#Hope it helps
I have had this question for a while.
Assuming I have a simple array,
my #arr;
foreach my $i (0..$#arr){
print "\$arr[$i] = '$arr[$i]'\n";
}
$#arr simply returns the last index of the array
however, what happens if I want to use it for something more complex like?:
foreach my $i (0..$##{$someHash{$}[$b]{$c}});
sure, I can write it as
foreach my $i (0..(scalar(#{$someHash{$}[$b]{$c}})-1));
But that just makes the code even more complex.
So my question is, can the $# be used for anything else other than a simple array( references, a list returned from a function call, etc...) and if so, how?
Thanks
Yes, you want:
for my $i (0..$#{$some_hash{$a}[$b]{$c}}) { ... }
If possible, though, I generally prefer
for my $value (#{ $some_hash{$a}[$b]{$c} }) { ... }
an index value is rarely as useful as the value stored in the array. Typically, if I need both, I add a counter instead:
my $i = 0;
for my $value (#{ $some_hash{$a}[$b]{$c} }) { ... } continue { $i++ }
and for clarity, I will also pre-assign the complex variable in the loop:
my $i = 0;
my $widget_list = $some_hash{$a}[$b]{$c};
for my $value (#{ $widget_list }) { ... } continue { $i++ }
Just because you can do some things in one line in Perl doesn't necessarily mean you should.
Yes, it can be done:
sub foo {
return { 'bar' => { 'array' => [ 'qux', [qw/ has three elements /] ] } };
}
print $#{ foo->{'bar'}->{'array'}->[1] }; # 2
When writing such code, my thought process is: get to the array with the #{} syntax and then replace # with $#.
If it's
$#array
for an array, it's
$#{ $array_ref }
for a reference. Of course, it can be any expression that returns an array reference, so you want the following:
$#{ $someHash{$}[$b]{$c} }
See Mini-Tutorial: Dereferencing Syntax
If you have a hash (or reference to a hash) in perl with many dimensions and you want to iterate across all values, what's the best way to do it. In other words, if we have
$f->{$x}{$y}, I want something like
foreach ($x, $y) (deep_keys %{$f})
{
}
instead of
foreach $x (keys %f)
{
foreach $y (keys %{$f->{$x})
{
}
}
Stage one: don't reinvent the wheel :)
A quick search on CPAN throws up the incredibly useful Data::Walk. Define a subroutine to process each node, and you're sorted
use Data::Walk;
my $data = { # some complex hash/array mess };
sub process {
print "current node $_\n";
}
walk \&process, $data;
And Bob's your uncle. Note that if you want to pass it a hash to walk, you'll need to pass a reference to it (see perldoc perlref), as follows (otherwise it'll try and process your hash keys as well!):
walk \&process, \%hash;
For a more comprehensive solution (but harder to find at first glance in CPAN), use Data::Visitor::Callback or its parent module - this has the advantage of giving you finer control of what you do, and (just for extra street cred) is written using Moose.
Here's an option. This works for arbitrarily deep hashes:
sub deep_keys_foreach
{
my ($hashref, $code, $args) = #_;
while (my ($k, $v) = each(%$hashref)) {
my #newargs = defined($args) ? #$args : ();
push(#newargs, $k);
if (ref($v) eq 'HASH') {
deep_keys_foreach($v, $code, \#newargs);
}
else {
$code->(#newargs);
}
}
}
deep_keys_foreach($f, sub {
my ($k1, $k2) = #_;
print "inside deep_keys, k1=$k1, k2=$k2\n";
});
This sounds to me as if Data::Diver or Data::Visitor are good approaches for you.
Keep in mind that Perl lists and hashes do not have dimensions and so cannot be multidimensional. What you can have is a hash item that is set to reference another hash or list. This can be used to create fake multidimensional structures.
Once you realize this, things become easy. For example:
sub f($) {
my $x = shift;
if( ref $x eq 'HASH' ) {
foreach( values %$x ) {
f($_);
}
} elsif( ref $x eq 'ARRAY' ) {
foreach( #$x ) {
f($_);
}
}
}
Add whatever else needs to be done besides traversing the structure, of course.
One nifty way to do what you need is to pass a code reference to be called from inside f. By using sub prototyping you could even make the calls look like Perl's grep and map functions.
You can also fudge multi-dimensional arrays if you always have all of the key values, or you just don't need to access the individual levels as separate arrays:
$arr{"foo",1} = "one";
$arr{"bar",2} = "two";
while(($key, $value) = each(%arr))
{
#keyValues = split($;, $key);
print "key = [", join(",", #keyValues), "] : value = [", $value, "]\n";
}
This uses the subscript separator "$;" as the separator for multiple values in the key.
There's no way to get the semantics you describe because foreach iterates over a list one element at a time. You'd have to have deep_keys return a LoL (list of lists) instead. Even that doesn't work in the general case of an arbitrary data structure. There could be varying levels of sub-hashes, some of the levels could be ARRAY refs, etc.
The Perlish way of doing this would be to write a function that can walk an arbitrary data structure and apply a callback at each "leaf" (that is, non-reference value). bmdhacks' answer is a starting point. The exact function would vary depending one what you wanted to do at each level. It's pretty straightforward if all you care about is the leaf values. Things get more complicated if you care about the keys, indices, etc. that got you to the leaf.
It's easy enough if all you want to do is operate on values, but if you want to operate on keys, you need specifications of how levels will be recoverable.
a. For instance, you could specify keys as "$level1_key.$level2_key.$level3_key"--or any separator, representing the levels.
b. Or you could have a list of keys.
I recommend the latter.
Level can be understood by #$key_stack
and the most local key is $key_stack->[-1].
The path can be reconstructed by: join( '.', #$key\_stack )
Code:
use constant EMPTY_ARRAY => [];
use strict;
use Scalar::Util qw<reftype>;
sub deep_keys (\%) {
sub deeper_keys {
my ( $key_ref, $hash_ref ) = #_;
return [ $key_ref, $hash_ref ] if reftype( $hash_ref ) ne 'HASH';
my #results;
while ( my ( $key, $value ) = each %$hash_ref ) {
my $k = [ #{ $key_ref || EMPTY_ARRAY }, $key ];
push #results, deeper_keys( $k, $value );
}
return #results;
}
return deeper_keys( undef, shift );
}
foreach my $kv_pair ( deep_keys %$f ) {
my ( $key_stack, $value ) = #_;
...
}
This has been tested in Perl 5.10.
If you are working with tree data going more than two levels deep, and you find yourself wanting to walk that tree, you should first consider that you are going to make a lot of extra work for yourself if you plan on reimplementing everything you need to do manually on hashes of hashes of hashes when there are a lot of good alternatives available (search CPAN for "Tree").
Not knowing what your data requirements actually are, I'm going to blindly point you at a tutorial for Tree::DAG_Node to get you started.
That said, Axeman is correct, a hashwalk is most easily done with recursion. Here's an example to get you started if you feel you absolutely must solve your problem with hashes of hashes of hashes:
#!/usr/bin/perl
use strict;
use warnings;
my %hash = (
"toplevel-1" =>
{
"sublevel1a" => "value-1a",
"sublevel1b" => "value-1b"
},
"toplevel-2" =>
{
"sublevel1c" =>
{
"value-1c.1" => "replacement-1c.1",
"value-1c.2" => "replacement-1c.2"
},
"sublevel1d" => "value-1d"
}
);
hashwalk( \%hash );
sub hashwalk
{
my ($element) = #_;
if( ref($element) =~ /HASH/ )
{
foreach my $key (keys %$element)
{
print $key," => \n";
hashwalk($$element{$key});
}
}
else
{
print $element,"\n";
}
}
It will output:
toplevel-2 =>
sublevel1d =>
value-1d
sublevel1c =>
value-1c.2 =>
replacement-1c.2
value-1c.1 =>
replacement-1c.1
toplevel-1 =>
sublevel1a =>
value-1a
sublevel1b =>
value-1b
Note that you CAN NOT predict in what order the hash elements will be traversed unless you tie the hash via Tie::IxHash or similar — again, if you're going to go through that much work, I recommend a tree module.