Perl populating a hash from an array of hashes - perl

I have a script where I am trying to populate a perl hash
I can dereference them fine when I do it individually
while(my($key,$value) = each(%{$spec_hash{'XISX'}})) {
print $key, "," .$value ;
print "\n";
}
while(my($key,$value) = each(%{$spec_hash{'XCBO'}})) {
print $key, "," .$value ;
print "\n";
}
However when i just try and dereference the %spec_hash It only containst one $exch reference, while it should had two - the XISX and the XCBO.
But it never gets to the XCBO.
#!/sbcimp/dyn/data/EVT/GSD/scripts/perl/bin/perl
use FOOConf; # this is our custom DBI module
use Data::Dumper ;
FOOConf::makeDBConnection(production);
my $dbh=$FOOConf::dbh;
my $query = "select e_risk_symbol from gsd_etds where level_name='EXCH_CS' and e_exch_dest='XISX' and e_symbol_comment in ('Bin_6','Bin_56')";
if(!$dbh) {
print "Error connecting to DataBase; $DBI::errstr\n";
}
my $cur_msg = $dbh->prepare($query) or die "\n\nCould not prepare statement:".$dbh->errstr;
$cur_msg->execute();
while (my #row=$cur_msg->fetchrow_array) {
$spec_hash{'XISX'}{$row[0]}=1;
}
$query = "select e_risk_symbol from gsd_etds where level_name='EXCH_CS' and e_exch_dest='XCBO' and e_combo_type='9999'";
if(!$dbh) {
print "Error connecting to DataBase; $DBI::errstr\n";
}
$cur_msg = $dbh->prepare($query) or die "\n\nCould not prepare statement: ".$dbh->errstr;
$cur_msg->execute();
while (my #row=$cur_msg->fetchrow_array) {
$spec_hash{'XCBO'}{$row[0]}=1;
}
#while(my($key,$value) = each(%spec_hash)) {
# print $key, "," .$value ;
# print "\n";
# }
#
# foreach my $exch (sort keys %spec_hash) {
# print "$exch: $spec_hash{$exch}" ;
# }
print Dumper(\%spec_hash);
this is the dumper - shouldn't the dumper contain the XCBO as well?
Why does the hash only have the XISX elements?
$VAR1 = {
'XISX' => {
'FCEL' => 1,
'GPS' => 1,
'MCO' => 1,
'DPZ' => 1,
'WM' => 1,
'SPLS' => 1,
'ILMN' => 1,
'BWLD' => 1,
'CTSH' => 1,
'EWU' => 1,
'MDVN' => 1,
'PDCO' => 1,
'AFAM' => 1,
'SHW' => 1,
}
};

Are you sure that you are populating it with those values?
Try adding a print statement in the while loop, something like this:
while (my #row=$cur_msg->fetchrow_array) {
$spec_hash{'XCBO'}{$row[0]}=1;
print "DEBUG $row[0]\n";
}
My guess is that your query is not returning any results to add to the hash. Unless I missed something, your other code looks fine.

Related

printing out a multilevel hash perl 5

I have to make sense of this script , without making major changes and insulting the guy who made it. I can't change the hashes, even though it would be easy to load data into arrays and then split. The guy who wrote this (my boss) loves his multilevel hashes. I have to print out the %extend_hash multilevel hash - and I don't understand how to get to the last level. I need to print this out in to a CSV file that cane be read by salespeople.
It looks like it goes out like 6 levels.
I have to sort keys of a hash, of a hash, of a hash ...etc.
#!/scripts/perl/bin/perl
use strict;
use warnings;
use DBI;
my $dbUser = 'foo_01';
my $dbPass = 'foo_01';
my $dbSid = 'foo.WORLD';
my $dbh = DBI->connect("dbi:Oracle:$dbSid","$dbUser","$dbPass") or die( "Couldn't connect: $!" );
#sub read_extend
my %extend_hash = ();
my $query = "select level_id,e_risk_symbol,e_exch_dest,penny,specialist from etds_extend";
if(!$dbh) {
print "Error connecting to DataBase; $DBI::errstr\n";
}
my $cur_msg = $dbh->prepare($query) or die "\n\nCould not prepare statement: ".$dbh->errstr;
$cur_msg->execute();
while (my #row=$cur_msg->fetchrow_array) {
$extend_hash{$row[0]}{$row[1]}{$row[2]}{'penny'}=$row[3];
$extend_hash{$row[0]}{$row[1]}{$row[2]}{'specialist'}=$row[4];
}
for my $what_row0 (sort keys %extend_hash) {
for my $what_row1 (sort keys %{$extend_hash {$what_row0} }) {
for my $what_row2 (sort keys ..... I am lost.
I don't know how to print out the %extend_hash down to the lowest level
I am trying to make it comma delimited, able to be pumped into an email and read by salespeople.
6,ACI,ARCX,specialist,1
6,ACI,ARCX,penny,0
6,MCHP,ARCX,specialist,1,
6,MCHP,ARCX,penny,0
6,BC,AMXO,specialist,1
6,BC,AMXO,penny,0
6,WM,XISX,specialist,1
6,WM,XISX,penny,0
6,PK,AMXO,specialist,1
6,PK,AMXO,penny,0
6,SPLS,XISX,specialist,1
6,SPLS,XISX,penny,0
If I use Data::Dumper I get this which is great, but the sales/marketing guys will get confused. They will not be able to see ARCX penny0 in realtions to the group '6'. I don't think they are able to mentally walt through the data::dump
$VAR1 = {
'6' => {
'IACI' => {
'ARCX' => {
'specialist' => '1',
'penny' => '0'
}
},
'MCHP' => {
'ARCX' => {
'specialist' => '1',
'penny' => '0'
}
},
'BC' => {
'AMXO' => {
'specialist' => '1',
'penny' => '0'
}
},
'WM' => {
'XISX' => {
'specialist' => '1',
'penny' => '0'
}
},
'PKD' => {
'AMXO' => {
'specialist' => '1',
'penny' => '0'
}
},
'SPLS' => {
'XISX' => {
'specialist' => '1',
'penny' => '0'
}
}
}
};
update - amazing work I_alarmed_alien - this should be a good stackoverflow reference (my comments are disabled)
for my $level_1 (sort keys %extend_hash) {
for my $level_2 (sort keys %{$extend_hash{$level_1} }) {
for my $level_3 (sort keys %{$extend_hash{$level_1}{$level_2}}) {
for my $type (sort keys %{$extend_hash{$level_1}{$level_2}{$level_3}} ) {
print "$level_1, $level_2, $level_3 $type" . " $extend_hash{$level_1}{$level_2}{$level_3}{$type}" ."\n" ;
}
}
}
}
got it - I_alarmed_alien amazing work outputing a hash of a hash of a hash of a hash of a hash of a hash
6, XLNX, AMXO , specialist 1
6, XLP, AMXO , penny 0
6, XLP, AMXO , specialist 1
6, XLP, XISX , penny 0
6, XLP, XISX , specialist 1
6, XLV, AMXO , penny 0
6, XLV, AMXO , specialist 1
6, XLY, AMXO , penny 0
6, XLY, AMXO , specialist 1
6, YUM, AMXO , penny 0
6, YUM, AMXO , specialist 1
6, ZINC, XISX , penny 0
6, ZINC, XISX , specialist 1
6, ZMH, AMXO , penny 0
6, ZMH, AMXO , specialist 1
to traverse the whole structure and create the comma-separated table (quick and dirty, but working solution):
my $VAR1 = <insert the hash here>
sub printval
{
my ($val, $path) = #_;
if (ref($val) eq "HASH")
{
printval ($val->{$_}, ($path?"$path,":"")."$_") foreach (keys %{$val});
}
else
{
print "$path,$val\n";
}
}
printval($VAR1);
You're almost there with your code. Here is how to get to the bottom of the hash:
foreach my $l1 (keys %extend_hash) {
# '6'
foreach my $l2 (keys %{$extend_hash{$l1}}) {
# IACI, MCHP, BC, etc.
foreach my $l3 (keys %{$extend_hash{$l1}{$l2}}) {
# ARCX, AMXO, XISX, etc.
foreach my $k (keys %{$extend_hash{$l1}{$l2}{$l3}}) {
print "$l1, $l2, $l3, $k, " . $extend_hash{$l1}{$l2}{$l3}{$k} . "\n";
}
}
}
}
Note that the hash keys are not accessed in any particular order, so you may want to sort them -- e.g. foreach my $l1 (sort keys %extend_hash).
Hashes of hashes of hashes of hashes are fun! ;)
ETA: Here is a more generic function for recursing into arbitrarily deep hashes-of-hashes-of-hashes-of...
sub print_hash {
# href = reference to the hash we're examining (i.e. \%extend_hash)
# so_far = arrayref containing the hash keys we are accessing
my $href = shift;
my $so_far = shift;
foreach my $k (keys %$href) {
# put $k on to the array of keys
push #$so_far, $k;
# if $href->{$k} is a reference to another hash, call print_hash on that hash
if (ref($href->{$k}) eq 'HASH') {
print_hash($href->{$k}, $so_far);
} else {
# $href->{$k} is a scalar, so print out #$so_far (our list of hash keys)
# and the value in $href->{$k}
print join(", ", #$so_far, $href->{$k}) . "\n";
}
# we've finished looking at $href->{$k}, so remove $k from the array of keys
pop #$so_far;
}
}
print_hash($hash, []);
if you only want to print it, use Data::Dumper http://perldoc.perl.org/Data/Dumper.html
To check whether a certain variable is a hash reference, use:
if (ref($r) eq "HASH") {
print "r is a reference to a hash.\n";
}
use Data::Dumper;
print Dumper %extend_hash;

dereferencing a multi-tiered hash in Perl

I cannot dereference the %spec_hash
Is it a multilevel hash?
#!/perl/bin/perl
use FOOConf; #custom module
use Data::Dumper ;
FOOConf::makeDBConnection(production);
use strict;
use warnings;
my $dbh=$EVTConf::dbh;
my $query = "select e_risk_symbol from gsd_etds where level_name='EXCH_CS' and e_exch_dest='XISX' and e_symbol_comment in ('Bin_6','Bin_56')";
if(!$dbh) {
print "Error connecting to DataBase; $DBI::errstr\n";
}
my $cur_msg = $dbh->prepare($query) or die "\n\nCould not prepare statement:".$dbh->errstr;
$cur_msg->execute();
while (my #row=$cur_msg->fetchrow_array) {
$spec_hash{'XISX'}{$row[0]}=1;
}
while(($key,$value) = each(%spech_hash)) {
print $key. "," .$value ;
}
I can see what is going into hash :
#!/perl/bin/perl
use strict;
use warnings ;
use FOOConf; # custom module we use for db access.
FOOConf::makeDBConnection(production); # amkes a database connection.
my $dbh=$EVTConf::dbh;
my $query = "select e_risk_symbol from gsd_etds where level_name='EXCH_CS' and e_exch_dest='XISX' and e_symbol_comment in ('Bin_6','Bin_56')";
if(!$dbh) {
print "Error connecting to DataBase; $DBI::errstr\n";
}
my $cur_msg = $dbh->prepare($query) or die "\n\nCould not prepare statement:".$dbh->errstr;
$cur_msg->execute();
while (my #row=$cur_msg->fetchrow_array) {
foreach $row(#row) {
print "$row ";
}
}
print "\n";
this is what i get ;
MTG GPS WM JBL ISIL MBI BA ILMN FCEL NDAQ CMS HOLX
INTC CYBX STLD MDT CTSH ASBC AMP KLAC LXK X MON
SYY HIG UNM AMGN STZ KMP SONC ECA BEBE EAT PLCE
SPN LAMR PDCO XLP GME CSGP EXC BHP
I can see what is getting loaded in to the $spec_hash by walking through the
#row array, and I can dump the %spec_hash
while (my #row=$cur_msg->fetchrow_array) {
$spec_hash{'XISX'}{$row[0]}=1;
}
print Dumper(%spec_hash)
In the dump is XISX the name of the hash and FCEL the key in hash XISX and the value of key FCEL is 1
foo#fooserver:/tmp/walt $ ./just_db.row.dumper
$VAR1 = 'XISX';
$VAR2 = {
'FCEL' => 1,
'GPS' => 1,
'MCO' => 1,
'DPZ' => 1,
'WM' => 1,
'SPLS' => 1,
'ILMN' => 1,
'BWLD' => 1,
'CTSH' => 1,
'EWU' => 1,
'MDVN' => 1,
'PDCO' => 1,
'AFAM' => 1,
'SHW' => 1,
I just can't dereference
"$spec_hash{'XISX'}{$row[0]}=1;"
there is somthing I don't understand about the way this hash is loaded.
Do this dump to see a better hash structure:
print Dumper(\%spec_hash);
To dereference you need:
while(my($key,$value) = each(%{$spech_hash{'XISX'}})) {

Get value from hash of hashes

I would like to get value from hash of hashes but i do not. My code is :
sub test {
my $filename = $_[0];
open INFILE, ${filename} or die $!;
my %hashCount;
my #firstline = split('\t',<INFILE>);
shift(#firstline);
while (my $line = <INFILE>)
{
my %temp;
chomp($line);
my #line = split('\t', $line);
foreach my $cpt (1..$#line) {
$temp{$firstline[$cpt-1]}=$line[$cpt];
}
$hashCount{$line[0]}={%temp};
}
return %hashCount;
}
sub get_hash_of_hash {
my $h = shift;
foreach my $key (keys %$h) {
if( ref $h->{$key}) {
get_hash_of_hash( $h->{$key} );
}
else {
say $h->{$key};
}
}
}
And when i display my hash :
$VAR10679 = 'M00967_43_1106_2493_14707';
$VAR10680 = {
'A' => '1',
'B' => '0',
'C' => '1',
'D' => '0',
'E' => '0'
};
My first function return my hash of hashes and i get my specific value with the second function.
So I want to get value like that :
my %hashTest = test("FILE.txt");
get_hash_of_hash(%hashTest,"M00967_43_1106_2493_14707","A")
//return value '1'
You can either access nested elements like
$hash{keyA}{keyB}
or we can write a function that walks the data structure, like
sub walk {
my ($hashref, #keys) = #_;
my $pointer = $hashref;
for my $key (#keys) {
if (exists $pointer->{$key}) {
$pointer = $pointer->{$key};
} else {
die "No value at ", join "->", #keys;
}
}
return $pointer;
}
which can be used like
my %hash = (
'M00967_43_1106_2493_14707' => {
'A' => '1',
'B' => '0',
'C' => '1',
'D' => '0',
'E' => '0'
},
);
say walk(\%hash, 'M00967_43_1106_2493_14707', 'A');
Note: When using Data::Dumper, pass references to the Dump function:
print Dump \%hash; # not print Dump %hash
This is neccessary to show the correct data structure.
Your hash holds references to hashes.
You can access them like this:
$hashTest{'M00967_43_1106_2493_14707'}{'A'};
See perlref for more info
Use this subroutine..
sub get_hash_of_hash {
my $h = shift;
foreach my $key (keys %$h) {
if( ref $h->{$key}) {
get_hash_of_hash( $h->{$key} );
}
else {
print $h->{$key};
}
}
}

Dereferencing on an Array of Hashes in Perl?

The data is stored in #emailaddresses as follows:
$VAR1 = { 'email' => 'abc#google.com' };
$VAR2 = { 'email' => 'cde#google.com' };
$VAR3 = { 'email' => 'efg#google.com' };
$VAR4 = { 'email' => 'hij#google.com' };
When I print #emailaddresses (print #emailaddresses), it prints HASH(0x...) instead of the values in the array. How can I print the values?
Simply loop through your array and print out each addresses 'email' value:
my #emailaddresses = ...;
for my $addr ( #emailaddresses ) {
print $addr->{email}, "\n";
}
better yet, wrap this logic in a function:
sub print_email_addresses {
my #addresses = #_;
for my $addr ( #addresses ) {
print $addr->{email}, "\n";
}
}

Perl Working On Two Hash References

I would like to compare the values of two hash references.
The data dumper of my first hash is this:
$VAR1 = {
'42-MG-BA' => [
{
'chromosome' => '19',
'position' => '35770059',
'genotype' => 'TC'
},
{
'chromosome' => '2',
'position' => '68019584',
'genotype' => 'G'
},
{
'chromosome' => '16',
'position' => '9561557',
'genotype' => 'G'
},
And the second hash is similar to this but with more hashes in the array. I would like to compare the genotype of my first and second hash if the position and the choromosome matches.
map {print "$_= $cave_snp_list->{$_}->[0]->{chromosome}\n"}sort keys %$cave_snp_list;
map {print "$_= $geno_seq_list->{$_}->[0]->{chromosome}\n"}sort keys %$geno_seq_list;
I could do that for the first array of the hashes.
Could you help me in how to work for all the arrays?
This is my actual code in full
#!/software/bin/perl
use strict;
use warnings;
use Getopt::Long;
use Benchmark;
use Config::Config qw(Sequenom.ini);
useDatabase::Conn;
use Data::Dumper;
GetOptions("sam=s" => \my $sample);
my $geno_seq_list = getseqgenotypes($sample);
my $cave_snp_list = getcavemansnpfile($sample);
#print Dumper($geno_seq_list);
print scalar %$geno_seq_list, "\n";
foreach my $sam (keys %{$geno_seq_list}) {
my $seq_used = $geno_seq_list->{$sam};
my $cave_used = $cave_snp_list->{$sam};
print scalar(#$geno_seq_list->{$_}) if sort keys %$geno_seq_list, "\n";
print scalar(#$cave_used), "\n";
#foreach my $seq2com (# {$seq_used } ){
# foreach my $cave2com( # {$cave_used} ){
# print $seq2com->{chromosome},":" ,$cave2com->{chromosome},"\n";
# }
#}
map { print "$_= $cave_snp_list->{$_}->[0]->{chromosome}\n" } sort keys %$cave_snp_list;
map { print "$_= $geno_seq_list->{$_}->[0]->{chromosome}\n" } sort keys %$geno_seq_list;
}
sub getseqgenotypes {
my $snpconn;
my $gen_list = {};
$snpconn = Database::Conn->new('live');
$snpconn->addConnection(DBI->connect('dbi:Oracle:pssd.world', 'sn', 'ss', { RaiseError => 1, AutoCommit => 0 }),
'pssd');
#my $conn2 =Database::Conn->new('live');
#$conn2->addConnection(DBI->connect('dbi:Oracle:COSI.world','nst_owner','nst_owner', {RaiseError =>1 , AutoCommit=>0}),'nst');
my $id_ind = $snpconn->execute('snp::Sequenom::getIdIndforExomeSample', $sample);
my $genotype = $snpconn->executeArrRef('snp::Sequenom::getGenotypeCallsPosition', $id_ind);
foreach my $geno (#{$genotype}) {
push #{ $gen_list->{ $geno->[1] } }, {
chromosome => $geno->[2],
position => $geno->[3],
genotype => $geno->[4],
};
}
return ($gen_list);
} #end of sub getseqgenotypes
sub getcavemansnpfile {
my $nstconn;
my $caveman_list = {};
$nstconn = Database::Conn->new('live');
$nstconn->addConnection(
DBI->connect('dbi:Oracle:CANP.world', 'nst_owner', 'NST_OWNER', { RaiseError => 1, AutoCommit => 0 }), 'nst');
my $id_sample = $nstconn->execute('nst::Caveman::getSampleid', $sample);
#print "IDSample: $id_sample\n";
my $file_location = $nstconn->execute('nst::Caveman::getCaveManSNPSFile', $id_sample);
open(SNPFILE, "<$file_location") || die "Error: Cannot open the file $file_location:$!\n";
while (<SNPFILE>) {
chomp;
next if /^>/;
my #data = split;
my ($nor_geno, $tumor_geno) = split /\//, $data[5];
# array of hash
push #{ $caveman_list->{$sample} }, {
chromosome => $data[0],
position => $data[1],
genotype => $nor_geno,
};
} #end of while loop
close(SNPFILE);
return ($caveman_list);
}
The problem that I see is that you're constructing a tree for generic storage of data, when what you want is a graph, specific to the task. While you are constructing the record, you could also be constructing the part that groups data together. Below is just one example.
my %genotype_for;
my $record
= { chromosome => $data[0]
, position => $data[1]
, genotype => $nor_geno
};
push #{ $gen_list->{ $geno->[1] } }, $record;
# $genotype_for{ position }{ chromosome }{ name of array } = genotype code
$genotype_for{ $data[1] }{ $data[0] }{ $sample } = $nor_geno;
...
return ( $caveman_list, \%genotype_for );
In the main line, you receive them like so:
my ( $cave_snp_list, $geno_lookup ) = getcavemansnpfile( $sample );
This approach at least allows you to locate similar position and chromosome values. If you're going to do much with this, I might suggest an OO approach.
Update
Assuming that you wouldn't have to store the label, we could change the lookup to
$genotype_for{ $data[1] }{ $data[0] } = $nor_geno;
And then the comparison could be written:
foreach my $pos ( keys %$small_lookup ) {
next unless _HASH( my $sh = $small_lookup->{ $pos } )
and _HASH( my $lh = $large_lookup->{ $pos } )
;
foreach my $chrom ( keys %$sh ) {
next unless my $sc = $sh->{ $chrom }
and my $lc = $lh->{ $chrom }
;
print "$sc:$sc";
}
}
However, if you had limited use for the larger list, you could construct the specific case
and pass that in as a filter when creating the longer list.
Thus, in whichever loop creates the longer list, you could just go
...
next unless $sample{ $position }{ $chromosome };
my $record
= { chromosome => $chromosome
, position => $position
, genotype => $genotype
};
...