I have a bit of logic drain. I hope I can explain what I am missing and what I want in a coherent manner. Let me know if I have to add a bit more data or information.
I have an Excel spreadsheet which I am trying to load to a database. I have slurped the data into an array of hashes. The data in the array looks like this
$hash_of_excel = [
{
col1 => 'value1',
col2 => 'value2',
col3 => 'value3|value4',
col4 => 'value5|value6|value7',
},
{
col1 => 'value8',
col2 => 'value9',
col3 => 'value10|value11|value12',
col4 => 'value13|value14|value15',
},
{
col1 => 'value16|value17',
col2 => 'value19|value18',
col3 => 'value20',
col4 => 'value21',
}
]
I have a piece of code that walks this data structure to get the values
foreach my $results ( #$hash_of_excel ) {
for my $colname ( sort keys %$results ) {
my #array = split /\|/, $results->{$colname};
foreach my $value ( #array ) {
warn $results->{'col1'}, $results->{'col2'}, $results->{'col3'};
last;
}
}
last if $counter++ == 2;
}
This would result in the same value printing over and over for the number of columns present in each hash (ie 4 in our case).
How can I access different columns for the DBI insert but without having to go through lot of for loops?
Is there a way to check if the value has more than one value and pushing them to array instead of having to get all of them in an array?
Or is this good to hand the database inserts to a subroutine and pass just the required column values in an array?
It's not clear what exactly you want, but your innermost loop is weird: it iterates over #array with $value, but $value isn't used in it - that's why you're getting the same output for all the iterations.
The following loop outputs all the values instead:
foreach my $value (#array){
warn $value;
}
i.e. no $results, no last.
Related
Please advice.
for my $record (#item) {
for my $int (#$record){
# DEBUG( "DEBUG:: $record and $int");
my %data = ( $record , $int );
}
}
}
Record is like
abc ,china
abc ,japan
abc , italy
abc , singapore
print Dumper %data;
output :
abc , singapore
Now the issues is when I dump the output it shows me last record entry in hash table.May be because of unique key.
Kindly suggest.
Two problems:
You are recreating the hash in each iteration of the loop. The correct way would be
my %data;
for my $record (#item) {
for my $int (#$record){
$data{$record} = $int;
}
}
Hash keys must be unique. It's not possible to have a hash like
( abc => 'china',
abc => 'japan' )
You can use a hash of arrays, though. Just assign to it with
push #{ $data{$record} }, $int;
It will create the following structure:
( abc => [ 'china', 'japan', 'italy', 'singapore' ] )
my %Hash= (2012=> 1, 1982=>12, 2010=>0);
The has key and values need to be all on the same field name 'time' like an array
$mycollection->insert(
{
'field1' => $var1;
'field2' => $var2;
#right here I need to know how to add above hash key and values
# like below
#'time': ["2012.1","1982.12","2010.0"]
}
);
Any suggestions or ideas will be apprecieated. This can probably accomplished by doing series of update statements but I would like to accomplish this with one insert statement due to my requirement.
I suppose your %Hash variable is something like this:
my %Hash= (2012=> 1, 1982=>12, 2010=>0);
So your array "time" is build this way:
my #time = map { $_ . "." . $Hash{$_} } keys %Hash;
and finally:
$mycollection->insert({
'field1' => $var1,
'field2' => $var2,
'time' => \#time
});
I remember having problem with DBI method selectrow_array. When i wasn't tidy enough i got back from it not the value of the column i asked, but count of columns (or something unwanted, i can't recall exactly). Now i try to refactor some code and i want to make sure in every possible place, that i get back only expected value. So i try to avoid surprises and find out which the bad behaviour was. From DBI docs i read that this may be really be problematic situation:
If called in a scalar context for a statement handle that has more
than one column, it is undefined whether the driver will return the
value of the first column or the last. So don't do that. Also, in a
scalar context, an "undef" is returned if there are no more rows or if
an error occurred. That "undef" can't be distinguished from an "undef"
returned because the first field value was NULL. For these reasons
you should exercise some caution if you use "selectrow_array" in a
scalar context, or just don't do that.
Still i can't force selectrow_array to return anything but value of the col1 (that's it what i am expecting)
my $query = 'SELECT col1, col2, col3 FROM table WHERE id = 112233';
my ( $c ) = ( $dbh->selectrow_array( $query ) );
my $x = ask_from_db();
my $y = $dbh->selectrow_array( $query );
my $z = ( $dbh->selectrow_array( $query ) );
my #A = $dbh->selectrow_array( $query );
say "C: $c"; # C: col1
say "X: $x"; # X: col1
say "Y: $y"; # Y: col1
say "Z: $z"; # Z: col1
say "A: #A"; # A: col1 col2 col3
sub ask_from_db {
return $dbh->selectrow_array( $query );
}
Every way i ask above, gives me fine result. How should i run the query to get wrong result?
wrong result != col1 value
The difference in outcome will be based on the implementation of the driver.
wantarray ? #row : $row[0]
vs
wantarray ? #row : $row[-1]
You'd use to use a different driver to get a different outcome. That said, I imagine you'll have a hard time finding a driver that doesn't return the first.
If you want to be sure to get the first, use:
( $dbh->selectrow_array( $query ) )[0]
What the documentation means by "it is undefined whether the driver will return the value of the first column or the last" is that the column returned is defined by the database driver and not DBI.
So the Postgres driver may decide to always return the first column whereas the mysql driver may always return the last column, or the column returned might depend on the query.
So don't call selectrow_array is scalar context - always call it in list context:
my #row = $sth->selectrow_array($query)
and you'll avoid all of the issues that the documentation mentions.
When I fetch the data this way is it possible then to access the column names and the column types or do I need an explicit prepare to reach this?
use DBI;
my $dbh = DBI->connect( ... );
my $select = "...";
my #arguments = ( ... );
my $ref = $dbh->selectall_arrayref( $select, {}, #arguments, );
Update:
With prepare I would do it this way:
my $sth = $dbh->prepare( $select );
$sth->execute( #arguments );
my $col_names = $sth->{NAME};
my $col_types = $sth->{TYPE};
my $ref = $sth->fetchall_arrayref;
unshift #$ref, $col_names;
The best solution is to use prepare to get a statement handle, as you describe in the second part of your question. If you use selectall_hashref or selectall_arrayref, you don't get a statement handle, and have to query the column type information yourself via $dbh->column_info (docs):
my $sth = $dbh->column_info('','',$table,$column); # or $column='' for all
my $info = $sth->fetchall_arrayref({});
use Data::Dumper; print Dumper($info);
(specifically, the COLUMN_NAME and TYPE_NAME attributes).
However, this introduces a race condition if the table changes schema between the two queries.
Also, you may use selectall_arrayref with the Slice parameter to fetch all the columns into a hash ref, it needs no prepared statement and will return an array ref of the result set rows, with each rows columns the key's to a hash and the values are the column values. ie:
my $result = $dbh->selectall_arrayref( qq{
SELECT * FROM table WHERE condition = value
}, { Slice => {} }) or die "Error: ".$dbh->errstr;
$result = [
[0] = { column1 => 'column1Value', column2 => 'column2Value', etc...},
[1] = { column1 => 'column1Value', column2 => 'column2Value', etc...},
];
Making it easy to iterate over results.. ie:
for my $row ( #$results ){
print "$row->{column1Value}, $row->{column2Value}\n";
}
You can also specify which columns to extract but it's pretty useless due to the fact it's more efficient to do that in your SQL query syntax.
{ Slice => { column1Name => 1, column2Name => 1 } }
That would only return the values for column1Name and column2Name just like saying in your SQL:
SELECT column1Name, column2Name FROM table...
I'm writing a web app in Perl using Dancer framework. The database is in sqlite and I use DBI for database interaction.
I'm fine with select statements, but I wonder is there a way to count selected rows.
E.g. I have
get '/' => sub {
my $content = database->prepare(sprintf("SELECT * FROM content LIMIT %d",
$CONTNUM));
$content->execute;
print(Dumper($content->fetchall_arrayref));
};
How do I count all items in the result without issuing another query?
What I want to achieve this way is showing 30 items per page and knowing how many pages there would be. Of course I can run SELECT COUNT (*) foo bar, but it looks wrong and redundant to me. I'm looking for a more or less general, DRY and not too heavy on database way to do so.
Any SQL or Perl hack or a hint what should I read about would be appreciated.
// I know using string concatenation for querys is bad
You have to do it the hard way: one query to get the count and another to get your desired slice of the row set:
my $count = $database->prepare('SELECT COUNT(*) FROM content');
$count->execute();
my $n = $count->fetchall_arrayref()->[0][0];
my $content = $database->prepare('SELECT * FROM content LIMIT ?');
$content->execute($CONTNUM);
#...
Not too familiar with perl, but I assume you can just store the result of $content->fetchall_arrayref and retrieve the count from that array befor you print it.
[edit]
Something like
my $ref = $content->fetchall_arrayref;
my $count = scalar(#$ref);
Don't use sqlite myself but the following might work:
select * from table join (select count(*) from table);
Whether the above works or not the first thing I'd look for is scrollable cursors if you are going to page through results - I doubt sqlite has those. However, in DBI you can use fetchall_arrayref with a max_rows to fetch a "page" at a time. Just look up the example in the DBI docs under fetchall_arrayref - it is something like this:
my $rowcache = [];
while( my $row = ( shift(#$rowcache) || shift(#{$rowcache=$sth->fetchall_arrayref(undef,100)||[]}) )
) {
# do something here
}
UPDATE: Added what you'd get with selectall_hashref assuming the table is called content with one integer column called "a":
$ perl -le 'use DBI; my $h = DBI->connect("dbi:SQLite:dbname=fred.db"); my $r = $h->selectall_hashref(q/select * from content join (select count(*) as count from content)/, "a");use Data::Dumper;print Dumper($r);'
$VAR1 = {
'1' => {
'count' => '3',
'a' => '1'
},
'3' => {
'count' => '3',
'a' => '3'
},
'2' => {
'count' => '3',
'a' => '2'
}
};
If you want to know how many results there will be, as well as getting the results themselves, all in one query, then get the count as a new value:
SELECT COUNT(*) AS num_rows, * from Table WHERE ...
Now the row count will be the first column of every row of your resultset, so simply pop that off before presenting the data.