Shorten code: merge arrays from hashes - perl

I have a list of hashes and some of the hashes contain a key which provides an array itself.
my #cars = (
{ # empty car
name => "BMW",
},
{ # car with passengers
name => "Mercedes",
passengers => [qw(Paul Willy)],
},
...
)
It's pretty much like above but of course not with the stupid cars example:-)
Now I need to get a list of all "passengers" from all hashes including ones that do not even offer a passengers array.
In a 2nd step I need to retrieve unique entries from the list (actually the passengers are Perl object refs and I need every object once in the list)
At the moment I do:
my (#all, #passengers, %seen);
for(#cars) {
push #all, #{$_->{passengers}} if $_->{passengers};
}
#passengers = grep { ! $seen{$_} ++ } #all;
I would like to get rid of #all and throw the list of all passengers directly into grep.
Any suggestions?

my %seen;
my #passengers = grep { ! $seen{$_} ++ }
map { #{$_->{passengers} || []} } #cars;

It bugs me to create an array and a reference only to immediately get rid of it both (like cjm did), so I'd use
my %seen;
my #passengers =
grep !$seen{$_}++,
map $_ ? #$_ : (),
map $_->{passengers},
#cars;
or
my %seen;
my #passengers =
grep !$seen{$_}++,
map #{ $_->{passengers} },
grep $_->{passengers},
#cars;

Here's another variation. It uses List::MoreUtils::uniq. The %seen stuff is good to know, but unnecessary these days.
use List::MoreUtils qw<uniq>;
my #passengers
= sort uniq map { #$_ } grep { defined } map { $_->{passengers} } #cars
;
Of course, using my idiom of list_if, I would just do this:
my #passengers = sort uniq map { list_if( $_->{passengers} ) } #cars;
Where list_if is defined as:
sub list_if {
use Params::Util qw<_ARRAY _HASH>;
return unless my $cond = shift;
return unless my $ref
= #_ == 0 ? $cond
: #_ == 1 ? $_[0]
: \#_
;
return !ref( $ref ) ? $ref
: _ARRAY( $ref ) ? #$ref
: _HASH( $ref ) ? %$ref
: ()
;
}
It's been a useful idiom for cutting down on long-hand approaches to deciding whether to "stream" array and hash refs or not.

Related

Perl: How to grep with Hash Slices?

I am trying to sort out defined parameters from a complex hash, using hash slices. Hash slices are great because they avoid lots of foreach and if defined so they simplify syntax greatly.
However, I'm clearly not doing this correctly:
use DDP;
my %hash = (
'a' => # first patient
{
'age' => 9,
'BMI' => 20
},
'b' =>
{
'age' => 8
}
);
my %defined_patients = grep {defined $hash{$_}{'age', 'BMI'}} keys %hash;
p %defined_patients;
The above code gives an empty hash, when I want it to return just patient a.
This question is similar to "no autovivication" pragma fails with grep in Perl and I've based my code on it.
I've also tried
my #defined_patients = grep {defined $hash{$_}{'age', 'BMI'}} keys %hash;
but that doesn't work either.
How can I use hash slices to grep patients with the defined keys?
If you want to check that none of the target keys are undefined, you have to check separately. This greps for undefined values:
grep { ! defined } #{ $hash{$_} }{ qw(age BMI) }
Notice that the hash slice
#{ $hash{$_} }{ qw(age BMI) }
In v5.24, you can use postfix dereferencing instead:
$hash{$_}->#{ qw(age BMI) }
But that grep has to fit in another one. Since you want the cases where all values are defined, you have to negate the result of the inner grep:
my #patients =
grep { ! grep { ! defined } $hash{$_}->#{ qw(age BMI) } }
keys %hash;
That's pretty ugly though. I'd probably do something simpler in a subroutine. This way you can handle any number of keys easily:
sub some_patients {
my( $hash, $keys ) = #_;
my #patient_keys;
foreach my $key ( keys %$hash ) {
next unless grep { ! defined } $hash{$key}->#{ #$keys };
push $key, #patient_keys;
}
return #patient_keys;
}
Now I simply call a subroutine instead of grokking multilevel greps:
my #patient_keys = some_patients( \%patients, [ qw(age BMI) ] );
Or, for something more targeted, maybe something like this that tests a particular sub-hash instead of the whole data structure:
sub has_defined_keys {
my( $hash, $keys ) = #_;
! grep { ! defined } $hash->#{#$keys}
}
my #target-keys = ...;
my #keys = grep {
has_defined_keys( $patients{$_}, \#target-keys )
} keys %patients;
Either way, when things start getting a bit too complex, use a subroutine to give those things names so you can hide the code in favor of something short.

how to declare array reference in hash refrence

my $memType = [];
my $portOp = [];
my $fo = "aster.out.DRAMA.READ.gz";
if($fo =~/aster.out\.(.*)\.(.*)\.gz/){
push (#{$memType},$1);
push (#{$portOp},$2);
}
print Dumper #{$memType};
foreach my $mem (keys %{$portCapability->{#{$memType}}}){
//How to use the array ref memType inside a hash//
print "entered here\n";
//cannot post the rest of the code for obvious reasons//
}
I am not able to enter the foreach loop . Can anyone help me fix it?
Sorry this is not the complete code . Please help me.
%{$portCapability->{#{$memType}}}
This doesn't do what you may think it means.
You treat $portCapability->{#{$memType}} as a hash reference.
The #{$memType} is evaluated in scalar context, thus giving the size of the array.
I aren't quite sure what you want, but would
%{ $portCapability->{ $memType->[0] } }
work?
If, however, you want to slice the elements in $portCapability, you would need somethink like
#{ $portCapability }{ #$memType }
This evaluates to a list of hashrefs. You can then loop over the hashrefs, and loop over the keys in an inner loop:
for my $hash (#{ $portCapability }{ #$memType }) {
for my $key (keys %$hash) {
...;
}
}
If you want a flat list of all keys of the inner hashes, but don't need the hashes themselves, you could shorten above code to
for my $key (map {keys %$_} #{ $portCapability }{ #$memType }) {
...;
}
I think what you want is this:
my $foo = {
asdf => {
a => 1, b => 2,
},
foo => {
c => 3, d => 4
},
bar => {
e => 5, f => 6
}
};
my #keys = qw( asdf foo );
foreach my $k ( map { keys %{ $foo->{$_} } } #keys ) {
say $k;
}
But you do not know which of these $k belongs to which key of $foo now.
There's no direct way to get the keys of multiple things at the same time. It doesn't matter if these things are hashrefs that are stored within the same hashref under different keys, or if they are seperate variables. What you have to do is build that list yourself, by looking at each of the things in turn. That's simply done with above map statement.
First, look at all the keys in $foo. Then for each of these, return the keys inside that element.
my $memType = [];
my $portOp = [];
my $fo = “aster.out.DRAMA.READ.gz”;
if ($fo =~ /aster.out\.(\w+)\.(\w+)\.gz/ ) { #This regular expression is safer
push (#$memType, $1);
push (#$portOp, $2);
}
print Dumper “#$memType”; #should print “DRAMA”
#Now if you have earlier in your program the hash %portCapability, your code can be:
foreach $mem (#$memType) {
print $portCapability{$mem};
}
#or if you have the hash $portCapability = {…}, your code can be:
foreach $mem (#$memType) {
print $portCapability->{$mem};
}
#Hope it helps

Delete an array element completely

Goal: Remove a particular Value from an array
I have written a script and it works fine but I am not happy the way I have written it. So I am curious to know is there a better way to write it. please consider below use case:
I have a nested hash/hash/array...like below. I need to remove any array values which has local in their name:
#!/usr/bin/perl -w
use strict;
use Data::Dumper;
my $hash = { esx1 =>
{ cluster => "clu1",
fd => "fd1",
ds => [
'ds1',
'ds2',
'localds',
],
},
esx2 =>
{ cluster => "clu2",
fd => "fd2",
ds => [
'ds3',
'ds4',
'dslocal',
],
},
};
foreach my $a ( keys %$hash )
{
foreach ( 0..$#{ $hash->{$a}->{ds} } )
{
delete $hash->{$a}->{ds}->[$_] if $hash->{$a}->{ds}->[$_] =~ /local/i;
#{ $hash->{$a}->{ds} } = grep defined, #{ $hash->{$a}->{ds} };
}
}
print Dumper ($hash);
so the script deletes the "localds" and "dslocal" and keeps everything else intact.
Question:
Is there a cleaner way to write the foreach ( 0..$#{$hash->{$a}->{ds} } ) loop
If I do not write the grep line above, the resultant array has the value containing local deleted but is replaced by undef. Why is this happening.
Thanks.
delete is for elements in hashes. It happens to work on arrays by a quirk of implementation, but it shouldn't be relied on.
For arrays, you want to use splice.
splice #{ $ref->{to}->{array} }, $index, 1, ();
This replaces the 1-element sublist starting at $index with (), the empty list.
Why first iterate through array and delete elements and then look for "not-deleted" nodes (side note - this grep should be outside loop)? You can look for good nodes from very start! Replace entire loop with:
foreach my $a ( keys %$hash )
{
#{ $hash->{$a}->{ds} } = grep { !/local/i } #{ $hash->{$a}->{ds} };
}
for my $h (values %$hash){
$h->{ds} = [ grep { $_ !~ /local/i } #{$h->{ds}} ];
}
Not much neater but:
foreach my $a ( keys %$hash )
{
my $temp;
foreach ( #{ $hash->{$a}->{ds} } )
{
push(#$temp, $_) unless $_ =~ /local/i;
}
$hash->{$a}->{ds} = $temp;
}
Delete doesn't alter the array structure, it just alters the array content. Because of this in your method you need to grep for defined entries to create a new array of the structure you desire, then overwrite the old array.
edit:
This is better explained on perldoc page for delete
delete() may also be used on arrays and array slices, but its behavior is less straightforward. Although exists() will return false for deleted entries, deleting array elements never changes indices of existing values; use shift() or splice() for that. However, if all deleted elements fall at the end of an array, the array's size shrinks to the position of the highest element that still tests true for exists(), or to 0 if none do.
edit:
As pointed out by mob splice will do what you want:
foreach my $a ( keys %$hash )
{
for(0..$#{ $hash->{$a}->{ds} } )
{
splice( #{ $hash->{$a}->{ds} }, $_, 1 ) if $hash->{$a}->{ds}->[ $_ ] =~ /local/i;
}
}
You could use List::MoreUtils qw/first_idx/ to get the index of /local/ like so
first_idx { $_ =~ /local/i } #{$hash->{$a}->{ds}};
Then do what you want with that.

In Perl, how can I implement map using grep?

Some time ago I was asked the “strange” question how would I implement map with grep.
Today I tried to do it, and here is what came out. Did I squeeze everything from Perl, or there are other more clever hacks?
#!/usr/bin/env perl
use strict;
use warnings;
use 5.010;
sub my_map(&#) {
grep { $_= $_[0]->($_) } #_[1..$#_];
}
my #arr = (1,2,3,4);
#list context
say (my_map sub {$_+1}, #arr);
#scalar context
say "".my_map {$_+1} #arr;
say "the array from outside: #arr";
say "builtin map:", (map {$_+1} #arr);
Are you sure they didn't ask how to implement grep with map? That's actually useful sometimes.
grep { STMTs; EXPR } LIST
can be written as
map { STMTs; EXPR ? $_ : () } LIST
(With one difference: grep returns lvalues, and map doesn't.)
Knowing this, one can compact
map { $_ => 1 } grep { defined } #list
to
map { defined ? $_ => 1 : () } #list
(I prefer the "uncompressed" version, but the "compressed" version is probably a little faster.)
As for implementing map using grep, well, you can take advantage of grep's looping and aliasing properties.
map { STMTs; EXPR } LIST
can be written as
my #rv;
grep { STMTs; push #rv, EXPR } LIST;
#rv
My attempt at this largely pointless academic exercise is.
sub my_map (&#) {
my $code = shift;
my #return_list;
grep {
push #return_list, $code->($_);
} #_;
return #return_list;
}
Using grep for this is a bit of a waste, because the return list of map may not be 1:1 with the input list, like my %hash = map { $_ => 1 } #array;, you need to be more generic that using the return list of grep. The result is any looking method that allows modifying the original list would work.
sub my_map (&#) {
my $code = shift;
my #return_list;
push #return_list, $code->($_) for #_;
return #return_list;
}
I'm not entirely sure what you mean (what aspect of mapist to be emulated by grep), but a typical map-scenario could be e.g. y = x**3:
...
my #list1 = (1,2,3,4,5);
my #list2 = map $_**3, #list1;
...
With grep, if you had to, you could almost make it looke like map (but destroying the original list):
...
my #list2 = grep { $_**=3; 1 } #list1;
...
by simply writing to the references of the original list elements. The problem here is the unwanted modification of the original list; this is what you don't want to do with map.
Therefore, we could just generate another list in a subroutine, modify this one and leave the original list untouched. In slight modification of Sinan Ünür's solution, this would read:
sub gap(&#) {
my $code = shift;
my #list = #_;
grep $_ = $code->($_), #list;
#list
}
my #arr = (1 .. 5);
# original map
print join ',', map { $_**3 } #arr;
# 1,8,27,64,125
# grep map
print join ',', gap { $_**3 } #arr;
# 1,8,27,64,125
# test original array
print join ',', #arr;
# 1,2,3,4,5 => untouched
Regards
rbo

How do you flatten a hash of key,value pairs?

I wanted to share with you guys a function I had created to see how I could optimize it, or if there was a better way to do this.
sub flatten{
my($ref,$delim,$item_delim,$array,$str) = #_;
die("Required Hash Reference") unless isHash($ref);
$delim = $delim ? $delim :'_';
#dump into array hash vals #simplified
if(!$item_delim){
#{$array} = %{$ref};
}else{
my($keys,$values);
$keys = getKeys($ref);
$values = getValues($ref);
#item strings
if($#$keys > 0 && $#$values > 0){
#fix for issue where value[n] is empty
#{$array}= map{ (defined $$values[ $_ ]) ? $$keys[ $_ ].$item_delim.$$values[ $_ ] : $$keys[ $_ ].$item_delim } 0 .. int($#$keys);
}else{
log "No Values to flatten";
return '';
}
}
$str = join($delim,#{$array});
return $str;
}
Are there any optimization points I should be aware of here?
Basically I want to go from
$HASH => {
key1 => 'val1',
key2 => 'val2',
key3 => 'val3',
}
to $STRING= key1=val1&key2=val2 ...
UPDATED
a solution without Modules is preferred I really just want to know how to effectively flatten a hash!.
Note that some of the functions here are simply wrapper functions that do what they say. isHash getKeys... pay no attention to those!
One convenient way is to use URI's query_form facility.
use URI;
my $uri = URI->new("", "http"); # We don't actually care about the path...
$uri->query_form(%params);
my $query_string = $uri->query;
Another, more manual way, is to just use URI::Escape, map, and join.
Without modules:
my $hashref = {
key1 => 'val1',
key2 => 'val2',
key3 => 'val3',
};
sub encode {
my $str = shift;
$str =~ s/([^A-Za-z0-9\.\/\_\-])/sprintf("%%%02X", ord($1))/seg;
return $str;
}
my $str = join '&' => map { encode($_).'='.encode($hashref->{$_}) } grep { defined $hashref->{$_} } keys %$hashref;
result:
key2=val2&key1=val1&key3=val3
I can't see anything in your question which means that your subroutine needs to be any more complex than:
sub flatten {
my ($hash, $delim, $item_delim) = #_;
$delim //= '&',
$item_delim //= '=';
return join $delim, map { "$_$item_delim$hash->{$_}" } keys %$hash;
}
Update: Getting a few downvotes here. I assume that people object to the fact that I'm not URI-encoding anything. I'll just point out that there's nothing in the original question saying that we're building URIs. If I knew that we were, then I'd certainly use the appropriate module.
This is why I said "I can't see anything in your question...".
use URI::Escape;
my $str=join '&',map {uri_escape($_).'='.uri_escape($QUERY_STRING->{$_})} grep {defined $QUERY_STRING->{$_}} keys %$QUERY_STRING;
I think this should work!