I'm trying to store the response from a Perl (v5.12.4) SOAP call (SQL query) as a hash for further processing while maintaining the structure of the data. The columns will vary so I'd like to be able to build the column keys dynamically. Here is an example of the reply:
$VAR1 = {
'row' => [
#0
{
'column1' => '',
'column2' => 'f',
'column3' => '0',
'column4' => '',
'column5' => 'f',
'column6' => '0',
'column7' => 'f',
},
]
};
I'm interested in the column names and values. I've made some (comical) attempts at handling this, such as:
unless ($res->fault) {
$Data::Dumper::Indent = 3;
my (%reply) = $res->paramsall();
foreach my $keys (keys %reply) {
print "$keys\n";
}
}
This gets a "Reference found where even-sized list expected at script.pl line...". Any point in the right direction here is much appreciated.
Thanks
This :
$res->paramsall()
almost certainy returns a hashref.
So what you are doing is assigning the hashref to the 1st key of %reply.
my %reply = %{ $res->paramsall() };
Will probably work.
You should take a look at perlreftut for some methods to work with refrerence structures.
Dereferencing hashes with my %hash = %{ } makes a copy of the referenced hash.
You can work directly with hashrefs like this :
my $reply = $res->paramsall();
foreach my $key1 (keys %$reply) {
foreach my $key2 (keys %{ $reply->{$key1} }) {
print " $key2: " . $reply->{$key1}{$key2} . "\n";
}
}
In the 1st code example in your reply, this :
$reply{$key2}->{$key2}
should read :
$reply{$key1}{$key2}
since everything is already dereferenced.
Related
I have an array of hashrefs built from a database using fethrow_hashref(). The data structure is built like so:
while (my $ref = $sth->fetchrow_hashref()) {
push #lines, $ref;
}
I sort the data in the query by program name ascending, so all of the references in the array are still in alphabetical order. Then, I go through each hash and find the value that is numerically equal to a '1'. I then take the caolumn name, and store it to compare to the rest of the hashrefs with that program name to ensure they all have a '1' in the same column.
my $pgm = "";
my $met_lvl = "";
my #devs = ();
my %errors = ();
my $error = "";
foreach my $line_ref (#lines) {
if ($pgm ne $line_ref->{"PROGRAM"}) {
if (#devs && $error) {
# print " Different number metal layers for $pgm: #devs \n";
$error = "";
}
#devs = ();
$pgm = $line_ref->{"PROGRAM"};
($met_lvl) = grep { $line_ref->{$_} == 1 } keys(%$line_ref);
push #devs, $line_ref->{"DEVICE"};
} elsif ($pgm eq $line_ref->{"PROGRAM"}) {
push #devs, $line_ref->{"DEVICE"};
my ($met_chk ) = grep { $line_ref->{$_} == 1 } keys(%$line_ref);
if ($met_chk ne $met_lvl) {
$errors{$line_ref->{"PROGRAM"}} = $line_ref->{"PROGRAM"};
$error = "YUP";
}
}
}
I'd like to be able to access the hashrefs individually, based on matching column names from the database. How can I access the hashrefs with "TEST" values for "PROGRAM" keys? I used Data::Dumper to provide an example of a few of the hashrefs I'd like to access based on "PROGRAM" value:
'PLM' => undef,
'SLM' => undef,
'QLM' => undef,
'DEVICE' => 'DEV1',
'TLM' => '1',
'DLM' => undef,
'ROUTING' => 'NORMAL',
'PROGRAM' => 'TEST'
};
$VAR455 = {
'PLM' => undef,
'SLM' => undef,
'QLM' => undef,
'DEVICE' => 'DEV2',
'TLM' => '1',
'DLM' => undef,
'ROUTING' => 'NORMAL',
'PROGRAM' => 'TEST'
};
$VAR456 = {
'PLM' => undef,
'SLM' => undef,
'QLM' => undef,
'DEVICE' => 'DEV3',
'TLM' => '1',
'DLM' => undef,
'ROUTING' => 'NON_STANDARD',
'PROGRAM' => 'EXP'
};
$VAR457 = {
'PLM' => undef,
'SLM' => undef,
'QLM' => undef,
'DEVICE' => 'DEV4',
'TLM' => '1',
'DLM' => undef,
'ROUTING' => 'NORMAL',
'PROGRAM' => 'FINAL'
};
I'd like to be able to access key values for the hashrefs which contain the same program name. I cannot even begin to figure out what type of operation to use for this. I assume map is the correct way to do it, but dereferencing the "PROGAM" value for each element (hashref) in the array is beyond the scope of my understanding. I hope I was able to define the problem well enough for you guys to be able to help.
Edit: The impetus for wanting to access hashrefs with the same 'PROGRAM" value is to be able to provide an output of selected values to print to a logfile. So, after I compare and find differences between those hashrefs with the same "PROGRAM" value, I want to access them all again, and print out the desired column values to the lofgile.
Looks like you need to exrtact subsets of your data (hashrefs) with the same PROGRAM name.
Can preprocess your data to build a hash with those names as keys, and arrayrefs (with suitable hashrefs) as values. Then process those groups one at a time.
use warnings;
use strict;
use feature 'say';
use Data::Dumper; # to print complex data below
... populate #lines with hashrefs as in the question or copy-paste a sample
# Build hash: ( TEST => [ hashrefs w/ TEST ], EXP => [ hashrefs w/ EXP ], ... )
my %prog_subset;
for my $hr (#lines) {
push #{ $prog_subset{$hr->{PROGRAM}} }, $hr;
# Or, using "postfix dereferencing" (stable from v5.24)
# push $prog_subset{$hr->{PROGRAM}}->#*, $hr;
}
foreach my $prog (keys %prog_subset) {
say "\nProcess hashrefs with PROGRAM being $prog";
foreach my $hr (#{ $prog_subset{$prog} }) {
say Dumper $hr;
}
}
(See postfix dereference)
Now %prog_subset contains keys TEST, EXP, FINAL (and whatever other PROGRAM names are in data), each having for value an arrayref of all hashrefs which have that PROGRAM name.
There are other ways, and there are libraries that can be leveraged, but this should do it.
OK! I found an example of this with the google machine. I replaced #lines = (); with $lines = [];. This allowed me to change the grep statement to (#found) = grep { $pgm eq $_->{PROGRAM} } #$lines;. Now the returned array is a list of the hashrefs that share the program name I'm looking for. Thanks for the help #zdim!
I totally got this question wrong. Am using the method from TMDB:
my #results = $search->find(id => 'tt0114694', source => 'imdb_id');
I thought the output was in JSON format, so that's what confused me, which kept me running in circles because I was looking at it all wrong.
Didn't realize the data below, from Dumper, was the actual hashes the I had to go through.
This is where I am running into a wall, So the data below is a hash with five keys. The fifth key, the I want, contains another array. It is that array I cannot read into. I try dereferencing that into a hash, and that is where I fail.
The code I am trying is:
foreach my $narray (#results){
print $narray->{"movie_results"};
my #newarray = $narray->{"movie_results"};
foreach my $otherarray (#newarray){
my %innerhash = $otherarray;
print %innerhash;
print "\n";
}
}
It will print out an array, but I am unable to read the hash in that array.
p.s. I had to format this output as code, or else it came out with no line breaks.
$VAR1 = {
'tv_season_results' => [],
'tv_results' => [],
'person_results' => [],
'tv_episode_results' => [],
'movie_results' => [
{
'adult' => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),
'vote_average' => '6.8',
'original_title' => 'Tommy Boy',
'vote_count' => 635,
'id' => 11381,
'release_date' => '1995-03-31',
'overview' => 'Party animal Tommy Callahan is a few cans short of a six-pack. But when the family business starts tanking, it\'s up to Tommy and number-cruncher Richard Hayden to save the day.',
'genre_ids' => [
35
],
'title' => 'Tommy Boy',
'video' => $VAR1->{'movie_results'}[0]{'adult'},
'poster_path' => '/g32WbO9nbY5ydpux5hIoiJkLEQi.jpg',
'original_language' => 'en',
'backdrop_path' => '/bZ4diYf7oyDVaRYeWG42Oify2mB.jpg',
'popularity' => '13.945'
}
]
};
You mention that you thought you'd get JSON output, but got something else. The module made a web request for you, received the JSON response, and translated that to a Perl data structure. That Perl version of the JSON is what you see in the dump.
A JSON object turns into a Perl hash, so that's what you see in the top level of the data structure. That's the single thing find returns (more on that in a moment):
Here's what you have, removing the outer foreach loop:
my #newarray = $narray->{"movie_results"};
foreach my $otherarray (#newarray){
my %innerhash = $otherarray;
print %innerhash;
print "\n";
}
The value in $narray->{"movie_results"} is an array reference. All references are scalars, and those scalars point to some data structure. When you assign that scalar to an array, you just end up with a one element array that's the same reference. Instead, you can
my $movie_results = $narray->{"movie_results"};
You then dereference that reference to treat it as an array:
foreach my $result ( #$movie_results ){ ... }
Or, the v5.24 postfix dereferencing way that I find slightly more pleasing since it reads better, especially when you skip the intermediate variable:
foreach my $result ( $movie_results->#* ){ ... }
foreach my $result ( $narray->{"movie_results"}->#* ){ ... }
That thing in $result is another hash reference.
References and data structures are about half of the content of Intermediate Perl, but there is also the Perl data structures cookbook (perldsc).
Improving your question a bit
You can help us a lot by showing us a complete, working demonstration of your problem. Here's what I cobbled together:
use v5.10;
use TMDB;
use Data::Dumper;
my $tmdb = TMDB->new( apikey => $ENV{TMDB_API_KEY} );
my #results = $tmdb->search->find(
id => 'tt0114694',
source => 'imdb_id'
);
say Dumper( \#results );
There was a question about the results of find. The documentation example shows it returning a list (well, the result being assigned to a named array, which implies that), but there's not actual documentation for find. It returns the decoded JSON from the response. Assigning it to a scalar (which will be a reference) works just fine too:
my $result = $tmdb->search->find(
id => 'tt0114694',
source => 'imdb_id'
);
say Dumper( $results );
The return value comes from TMDB::Sesssion::talk(), which is just this (or the empty list):
return $self->json->decode(
Encode::decode( 'utf-8-strict', $response->{content} ) );
That's not a big deal. That just means you don't need the outer foreach. That's not on you because the example in the docs tells you to do exactly what you did.
Now a better program
Putting all that together, here's a simple program pared down to just what you need:
use v5.10;
use TMDB;
my $tmdb = TMDB->new( apikey => $ENV{TMDB_API_KEY} );
my $result = $tmdb->search->find(
id => 'tt0114694',
source => 'imdb_id'
);
foreach my $item ( $result->{movie_results}->#* ) {
say "Title: $item->{title}";
}
Ref aliasing
There's an experimental feature called ref aliasing that lets you assign a reference to a reference of a named variable. It's an alias, so you can access and change the same data, just with a named variable. Something this is handy when you don't like
use v5.10;
use TMDB;
use experimental qw(refaliasing);
my $tmdb = TMDB->new( apikey => $ENV{TMDB_API_KEY} );
# response is a hash ref, so ref alias to a named hash
\my %result = $tmdb->search->find(
id => 'tt0114694',
source => 'imdb_id'
);
# the part you want is an array ref, so alias that
\my #movie_results = $result{movie_results};
# each item in the array is a hash ref, so alias those too
foreach \my %item ( #movie_results ) {
say "Title: $item{title}";
}
When dealing with reference, use the same syntax as if you weren't, but replace the name of the variable with a block that returns the reference.
%NAME -> %{ $ref } Or just %$ref
$NAME{...} -> ${ $ref }{...} Although $ref->{...} easier to read.
#NAME -> #{ $ref } Or just #$ref
$NAME[...] -> ${ $ref }[...] Although $ref->[...] easier to read.
Let's give $VAR a better name,
my $response = $VAR1;
This means you want
my $results = $response->{movie_results};
for my $result (#$results) {
for my $key (keys(%$result)) {
say "$key: $result->{$key}";
}
}
See
perlreftut
Perl Dereferencing Syntax
%newhash{$newkey} should be $newhash{$newkey}.
I'm trying to count starts and stops of some services i keep track of in logs.
I'm not going to past here entire code, but my way of doing hash is this:
I'm passing those starts and stops into anonymous hash .
First I'm creating anonymous hash filled with keys and values (in my case $knot is a key an zeros are values). Next im replaqcing values with another hash.
My code looks like this:
foreach $knot (#knots){
chomp $knot;
$variable = $variable."$knot;0;";
$Services = {split(/;/,$variable)};
}
my $data =
{
Starts=>'0',
Stops=>'0',
};
foreach my $key (keys %$Services) {
$Services->{$key} = $data;
}
print Dumper $Services;
Printing shows:
$VAR1 = {
' knot1' => {
'Stops' => '0',
'Starts' => '0'
},
' knot2' => $VAR1->{' knot1'},
' knot3' => $VAR1->{' knot1'},
' knot4' => $VAR1->{' knot1'},
' knot5' => $VAR1->{' knot1'},
and so on. Is there a better way of doing this? My way if i'm correct is badly written because changing knot1 starts/stops changes every other knot values.
Counting is very simple in Perl, thanks to Autovivification. You can just create anonymous data structures on the fly, like so:
use Data::Dumper;
my %hash = ();
$hash{apple}{green}++;
$hash{apple}{red} ++;
$hash{pear}{yellow}++;
$hash{apple}{green}++;
$hash{apple}{red} ++;
$hash{apple}{green}++;
print Dumper(\%hash);
This will produce the desired structure for counting:
$VAR1 = {
'apple' => {
'green' => 3,
'red' => 2
},
'pear' => {
'yellow' => 1
}
};
This also works in loops using variables (here using a reference to a hash):
my $hash_ref = {};
for my $fruit (qw( apple pear apple peach apple pear )) {
$hash_ref->{$fruit}++;
}
print Dumper($hash_ref);
resulting in:
$VAR1 = {
'peach' => 1,
'pear' => 2,
'apple' => 3
};
I have a data structure which looks like this:
my %hoh = (
'T431567' => {
machin => '01',
bidule => '02',
truc => '03',
},
'T123456' => {
machin => '97',
bidule => '99',
truc => '69',
},
'T444444' => {
machin => '12',
bidule => '64',
truc => '78',
},
);
I want to search the various values of truc for a particular value and find the top-level attribute which corresponds to that entry. For example, looking for a value of 78, I want to find the result 'T444444', because $hoh{T444444}{truc} is 78.
How can I do this, please?
You can do this with grep:
my #keys = grep { $hoh{$_}{truc} == 78 } keys %hoh;
Note that this can return more than one key, if there are duplicate values in the hash. Also note that this is not particularly efficient, since it has to search the entire hash. In most cases it's probably fine, but if the hash can be very large and you may need to run lots of such queries against it, it may be more efficient to build a reverse index as suggested by Sobrique:
my %trucs;
foreach my $part (keys %hoh) {
my $val = $hoh{$part}{truc};
push #{ $trucs{$val} }, $part;
}
my #keys = #{ $trucs{78} };
or, more generally:
my %index;
foreach my $part (keys %hoh) {
my %data = %{ $hoh{$part} };
foreach my $key (keys %data) {
my $val = $data{$key};
push #{ $index{$key}{$val} }, $part;
}
}
my #keys = #{ $index{truc}{78} };
Can't with that data structure as is - There is no 'backwards' relationship from value to key without you creating it.
You've two options - run a search, or create an 'index'. Practically speaking, these are the same, just one saves the results.
my %index;
foreach my $key ( keys %hoh ) {
my $truc = $hoh{$key}{'truc'};
$index{$truc} = $key;
}
Note - won't do anything clever if the 'truc' numbers are duplicated - it'll overwrite. (Handling this is left as an exercise to the reader).
This solution is similar to those already posted, but it uses the each operator to process the original hash in fewer lines of code, and probably more quickly.
I have added the dump output only so that you can see the form of the structure that is built.
use strict;
use warnings;
my %hoh = (
T123456 => { bidule => '99', machin => '97', truc => '69' },
T431567 => { bidule => '02', machin => '01', truc => '03' },
T444444 => { bidule => '64', machin => '12', truc => '78' },
);
my %trucs;
while ( my ($key, $val) = each %hoh ) {
next unless defined( my $truc = $val->{truc} );
push #{ $trucs{$truc} }, $key ;
}
use Data::Dump;
dd \%trucs;
print "\n";
print "$_\n" for #{ $trucs{78} };
output
{ "03" => ["T431567"], "69" => ["T123456"], "78" => ["T444444"] }
T444444
If you can guarantee that the answer is unique, i.e. that there is never more than one element of the original hash that has a given value for the truc entry, or you are interested only in the last one found, then you can write this still more neatly
my %trucs;
while ( my ($key, $val) = each %hoh ) {
next unless defined( my $truc = $val->{truc} );
$trucs{$truc} = $key ;
}
print $trucs{78}, "\n";
output
T444444
Simplest of all, if there is always a truc entry in each second-level hash, and its values is guaranteed to be unique, then this will do the job
my %trucs = map { $hoh{$_}{truc} => $_ } keys %hoh;
print $trucs{78}, "\n";
with the output as above.
I'm rather inexperienced with hashes of hashes - so I hope someone can help a newbie...
I have the following multi-level hash:
$OCRsimilar{$ifocus}{$theWord}{"form"} = $theWord;
$OCRsimilar{$ifocus}{$theWord}{"score"} = $OCRscore;
$OCRsimilar{$ifocus}{$theWord}{"distance"} = $distance;
$OCRsimilar{$ifocus}{$theWord}{"similarity"} = $similarity;
$OCRsimilar{$ifocus}{$theWord}{"length"} = $ilength;
$OCRsimilar{$ifocus}{$theWord}{"frequency"} = $OCRHashDict{$ikey}{$theWord};
Later, I need to sort each second-level element ($theWord) according to the score value. I've tried various things, but have failed so far. The problem seems to be that the sorting introduces new empty elements in the hash that mess things up.
What I have done (for example - I'm sure this is far from ideal):
my #flat = ();
foreach my $key1 (keys { $OCRsimilar{$ifocus} }) {
push #flat, [$key1, $OCRsimilar{$ifocus}{$key1}{'score'}];
}
for my $entry (sort { $b->[1] <=> $a->[1] } #flat) {
print STDERR "#$entry[0]\t#$entry[1]\n";
}
If I check things with Data::Dumper, the hash contains for example this:
'uroadcast' => {
'HASH(0x7f9739202b08)' => {},
'broadcast' => {
'frequency' => '44',
'length' => 9,
'score' => '26.4893274374278',
'form' => 'broadcast',
'distance' => 1,
'similarity' => 1
}
}
If I don't do the sorting, the hash is fine. What's going on?
Thanks in advance for any kind of pointers...!
Just tell sort what to sort on. No other tricks are needed.
#!/usr/bin/perl
use warnings;
use strict;
my %OCRsimilar = (
focus => {
word => {
form => 'word',
score => .2,
distance => 1,
similarity => 1,
length => 4,
frequency => 22,
},
another => {
form => 'another',
score => .01,
distance => 1,
similarity => 1,
length => 7,
frequency => 3,
},
});
for my $word (sort { $OCRsimilar{focus}{$a}{score} <=> $OCRsimilar{focus}{$b}{score} }
keys %{ $OCRsimilar{focus} }
) {
print "$word: $OCRsimilar{focus}{$word}{score}\n";
}
Pointers: perlreftut, perlref, sort.
What seems suspicious to me is this construct:
foreach my $key1 (keys { $OCRsimilar{$ifocus} }) {
Try dereferencing the hash, so it becomes:
foreach my $key1 (keys %{ $OCRsimilar{$ifocus} }) {
Otherwise, you seem to be creating an anonymous hash and taking the keys of it, equivalent to this code:
foreach my $key1 (keys { $OCRsimilar{$ifocus} => undef }) {
Thus, I think $key1 would equal $OCRsimilar{$ifocus} inside the loop. Then, I think Perl will do auto-vivification when it encounters $OCRsimilar{$ifocus}{$key1}, adding the hash reference $OCRsimilar{$ifocus} as a new key to itself.
If you use warnings;, the program ought to complain Odd number of elements in anonymous hash.
Still, I don't understand why Perl doesn't do further auto-vivication and add 'score' as the key, showing something like 'HASH(0x7f9739202b08)' => { 'score' => undef }, in the Data dump.