I'm writing a web app in Perl using Dancer framework. The database is in sqlite and I use DBI for database interaction.
I'm fine with select statements, but I wonder is there a way to count selected rows.
E.g. I have
get '/' => sub {
my $content = database->prepare(sprintf("SELECT * FROM content LIMIT %d",
$CONTNUM));
$content->execute;
print(Dumper($content->fetchall_arrayref));
};
How do I count all items in the result without issuing another query?
What I want to achieve this way is showing 30 items per page and knowing how many pages there would be. Of course I can run SELECT COUNT (*) foo bar, but it looks wrong and redundant to me. I'm looking for a more or less general, DRY and not too heavy on database way to do so.
Any SQL or Perl hack or a hint what should I read about would be appreciated.
// I know using string concatenation for querys is bad
You have to do it the hard way: one query to get the count and another to get your desired slice of the row set:
my $count = $database->prepare('SELECT COUNT(*) FROM content');
$count->execute();
my $n = $count->fetchall_arrayref()->[0][0];
my $content = $database->prepare('SELECT * FROM content LIMIT ?');
$content->execute($CONTNUM);
#...
Not too familiar with perl, but I assume you can just store the result of $content->fetchall_arrayref and retrieve the count from that array befor you print it.
[edit]
Something like
my $ref = $content->fetchall_arrayref;
my $count = scalar(#$ref);
Don't use sqlite myself but the following might work:
select * from table join (select count(*) from table);
Whether the above works or not the first thing I'd look for is scrollable cursors if you are going to page through results - I doubt sqlite has those. However, in DBI you can use fetchall_arrayref with a max_rows to fetch a "page" at a time. Just look up the example in the DBI docs under fetchall_arrayref - it is something like this:
my $rowcache = [];
while( my $row = ( shift(#$rowcache) || shift(#{$rowcache=$sth->fetchall_arrayref(undef,100)||[]}) )
) {
# do something here
}
UPDATE: Added what you'd get with selectall_hashref assuming the table is called content with one integer column called "a":
$ perl -le 'use DBI; my $h = DBI->connect("dbi:SQLite:dbname=fred.db"); my $r = $h->selectall_hashref(q/select * from content join (select count(*) as count from content)/, "a");use Data::Dumper;print Dumper($r);'
$VAR1 = {
'1' => {
'count' => '3',
'a' => '1'
},
'3' => {
'count' => '3',
'a' => '3'
},
'2' => {
'count' => '3',
'a' => '2'
}
};
If you want to know how many results there will be, as well as getting the results themselves, all in one query, then get the count as a new value:
SELECT COUNT(*) AS num_rows, * from Table WHERE ...
Now the row count will be the first column of every row of your resultset, so simply pop that off before presenting the data.
Related
I want to get the 1st row of the result depends on which build the room is. For example Building 1 have 1-200 rooms and Building 2 have 201-400 rooms. The code I tried is below. I have used the MIN in the where clause but I got all the rooms instead of having one.
$query = $this->db->query("SELECT * FROM `ha_utility_reading`");
if ($query->num_rows == 0) {
echo "some data match";
$lastroom = $this->db->select("*")->from("rooms")
->where("(SELECT MIN(room_num) FROM ha_rooms) and bldg_num = '$bldg_num'")
->get()->result_array();
foreach($lastroom as $key => $test) {
$output['room_num'][] = $test['room_num'];
json_encode($output);
}
You get all the rows because you need a group by clause. Anyway, the best way to do this is just adding this to your query:
order by room_num asc limit 1;
Try this,
select * from rooms order by room_num asc limit 1;
In my controller, I have the following code:
$threads = MessageThread::whereExists(function($query) {
$messid = '`' . env('DB_PREFIX') . 'message_threads`.`id`';
$query->select(\DB::raw(1))
->from('message_thread_participants')
->where('message_thread_participants.message_thread_id', '=', $messid); // AND message_thread_participants.user_id = ' . $user_id);
})
->orderBy('message_threads.created_at', 'DESC')
->take(30)
->get();
If I print out the resulting query using DB::getQueryLog() I get the following:
[ {
"query": "select * from bd45_message_threads where exists (select 1 from bd45_message_thread_participants where
bd45_message_thread_participants.message_thread_id = ?) order by
bd45_message_threads.created_at desc limit 30",
"bindings": [
"bd45_message_threads.id"
],
"time": 1.06 } ]
And I run that query manually (substituting the binding for the ?, I get the expected 30 results..... but the code itself returns no results... I assume it's some problem between using Models and Where Exists... but it's just a guess right now.... anyone have any clue why the query would return results in Navicat or PHPMyAdmin, but the controller returns a blank array?
I'm a little late to the party, but in case someone has the same problem, it must be beacause you have to use
->whereColumn('bd45_message_thread_participants.message_thread_id', 'bd45_message_threads.id')
to compare two columns values. The ->where expects to compare a column and a value, not another column.
I remember having problem with DBI method selectrow_array. When i wasn't tidy enough i got back from it not the value of the column i asked, but count of columns (or something unwanted, i can't recall exactly). Now i try to refactor some code and i want to make sure in every possible place, that i get back only expected value. So i try to avoid surprises and find out which the bad behaviour was. From DBI docs i read that this may be really be problematic situation:
If called in a scalar context for a statement handle that has more
than one column, it is undefined whether the driver will return the
value of the first column or the last. So don't do that. Also, in a
scalar context, an "undef" is returned if there are no more rows or if
an error occurred. That "undef" can't be distinguished from an "undef"
returned because the first field value was NULL. For these reasons
you should exercise some caution if you use "selectrow_array" in a
scalar context, or just don't do that.
Still i can't force selectrow_array to return anything but value of the col1 (that's it what i am expecting)
my $query = 'SELECT col1, col2, col3 FROM table WHERE id = 112233';
my ( $c ) = ( $dbh->selectrow_array( $query ) );
my $x = ask_from_db();
my $y = $dbh->selectrow_array( $query );
my $z = ( $dbh->selectrow_array( $query ) );
my #A = $dbh->selectrow_array( $query );
say "C: $c"; # C: col1
say "X: $x"; # X: col1
say "Y: $y"; # Y: col1
say "Z: $z"; # Z: col1
say "A: #A"; # A: col1 col2 col3
sub ask_from_db {
return $dbh->selectrow_array( $query );
}
Every way i ask above, gives me fine result. How should i run the query to get wrong result?
wrong result != col1 value
The difference in outcome will be based on the implementation of the driver.
wantarray ? #row : $row[0]
vs
wantarray ? #row : $row[-1]
You'd use to use a different driver to get a different outcome. That said, I imagine you'll have a hard time finding a driver that doesn't return the first.
If you want to be sure to get the first, use:
( $dbh->selectrow_array( $query ) )[0]
What the documentation means by "it is undefined whether the driver will return the value of the first column or the last" is that the column returned is defined by the database driver and not DBI.
So the Postgres driver may decide to always return the first column whereas the mysql driver may always return the last column, or the column returned might depend on the query.
So don't call selectrow_array is scalar context - always call it in list context:
my #row = $sth->selectrow_array($query)
and you'll avoid all of the issues that the documentation mentions.
I'm trying to use fetchAll on a query that has 2 variables. I can't figure out the syntax.
I can manage with only 1 variable:
$sql = "SELECT * FROM mytable WHERE field1 = ?";
$this->_db->fetchAll($sql,$value1); # that works
However I'm having some issues when query has multiple variables
$sql = "SELECT * FROM mytable WHERE field1 = ? AND field2 = ?";
$this->_db->fetchAll($sql,$value1,$value2); # doesn't work
$this->_db->fetchAll($sql,array("field1"=>$value1,"field2"=>$value2)); # doesn't work either
The reason why I want to use ? instead of placing the variables directly into the query is that I've learned that using ? allows for the query to be compiled generically by the db engine and improves performances.
There are two types of parameter, named parameters and positional parameters. You're mixing the two types and that won't work.
Named parameters match a placeholder by name. Names are started with the : symbol. The parameter names are not the same as the names of the columns you happen to use them for. You supply parameter values in an associative array, using the parameter name (not the column name) as the array keys. For example:
$sql = "SELECT * FROM mytable WHERE field1 = :param1 AND field2 = :param2";
$this->_db->fetchAll($sql,array("param1"=>$value1,"param2"=>$value2));
Positional parameters use the ? symbol for the placeholder. You supply parameter values using a simple (non-associative) array, and the order of values in the array must match the order of parameter placeholders in your query. For example:
$sql = "SELECT * FROM mytable WHERE field1 = ? AND field2 = ?";
$this->_db->fetchAll($sql,array($value1,$value2));
Most brands of SQL database natively support only one style or the other, but PDO attempts to support both, by rewriting the SQL if necessary before preparing the query. Since Zend_Db is modeled after PDO, Zend_Db also supports both parameter styles.
This question is a bit old, but I thought I'd just add to it for reference sake.
I would recommend starting to use Zend_Db_Select with Zend_Db. I've been doing a lot with Zend_Db lately. More from Zend_Db_Select reference guide.
Lets assume you have a Zend_Db adapter: $this->_db
# this will get the Zend_Db_Select object
$select = $this->_db->select();
# now you build up your query with Zend_Db_Select functions
$select->from('mytable');
$select->where('field1 = ?', $field1);
$select->where('field2 = ?', $field2);
[...]
# echo to see the SQL (helps in debugging)
# SELECT * FROM mytable WHERE field1 = ? AND field2 = ? [...]
echo '<p>My SQL: ' . $select . '</p>';
# Execute the SQL / Fetch results
$results = $select->query()->fetchAll();
That's the basics from your given example, but the Zend Framework reference guide on the select object has a lot of good information on how to build even more complex queries with JOINS, UNIONS, GROUP BY, LIMIT, HAVING, etc.
If you wanted to use an alias name for a table or parameters, you use an associative array with the alias name being the index value:
# SELECT p.* FROM products AS p
$select->from('p' => 'products');
If you want to return only selected fields, you add an array of field names as a second parameter:
# SELECT model FROM products
$select->from(products, array(model));
Actually, the above could should produce fully qualified SQL as:
SELECT 'products'.model FROM 'products'
but I wrote the above for brevity and clarity in the example.
One thing I just came across is using AND and OR in the WHERE condition.
# WHERE a = $a
$select->where('a = ?', $a);
# WHERE a = $a AND b = $b
$select->where('a = ?', $a);
$select->where('b = ?', $b);
# WHERE a = $a OR b = $b
$select->where('a = ?', $a);
$select->orWhere('b = ?', $b);
# WHERE a = $a AND b = $b
$select->orWhere('a = ?', $a);
$select->where('b = ?', $b);
Notice, that whatever the following "where" function you use, will combine with the previous statement as that operand. Ok, that sounded confusing.
If the second "where" is an "OR" it will be an "OR" conditional. If the second "where" is a "AND" the statement will be "AND".
In other words, the first WHERE function is ignored in terms of what condition it will use.
In fact, I just asked a question on Stack Overflow yesterday regarding doing a complex WHERE using select.
Hope that helps!
Cheers!
Try this:
$sql = "SELECT * FROM mytable WHERE field1 = ? AND field2 = ?";
$statement = $this->_db->query($sql,array("field1"=>$value1,"field2"=>$value2));
$data = $statement->fetchAll();
$this->_db must be an instance of Db adapter.
Heres the actual Zend way to code for this.
$sql = "SELECT * FROM mytable WHERE field1 = :param1 AND field2 = :param2";
$this->_db->fetchAll($sql,array("param1"=>$value1,"param2"=>$value2));
$where = $this->_db->select()
->from('mytable')
->where('field1 = ?',$value1)
->where('field2 = ?',$value2);
$rowSet = $this->_db->fetchAll($where);
This works great for me
I am writing small snippets in Perl and DBI (SQLite yay!)
I would like to log some specific queries to text files having the same filename as that of the table name(s) on which the query is run.
Here is the code I use to dump results to a text file :
sub dumpResultsToFile {
my ( $query ) = #_;
# Prepare and execute the query
my $sth = $dbh->prepare( $query );
$sth->execute();
# Open the output file
open FILE, ">results.txt" or die "Can't open results output file: $!";
# Dump the formatted results to the file
$sth->dump_results( 80, "\n", ", ", \*FILE );
# Close the output file
close FILE or die "Error closing result file: $!\n";
}
Here is how I can call this :
dumpResultsToFile ( <<" END_SQL" );
SELECT TADA.fileName, TADA.labelName
FROM TADA
END_SQL
What I effectively want is, instead of stuff going to "results.txt" ( that is hardcoded above ), it should now go to "TADA.txt".
Had this been a join between tables "HAI" and "LOL", then the resultset should be written to "HAI.LOL.txt"
Is what I am saying even possible using some magic in DBI?
I would rather do without parsing the SQL query for tables, but if there is a widely used and debugged function to grab source table names in a SQL query, that would work for me too.
What I want is just to have a filename
that gives some hint as to what query
output it holds. Seggregating based on
table name seems a nice way for now.
Probably not. Your SQL generation code takes the wrong approach. You are hiding too much information from your program. At some point, your program knows which table to select from. Instead of throwing that information away and embedding it inside an opaque SQL command, you should keep it around. Then your logger function doesn't have to guess where the log data should go; it knows.
Maybe this is clearer with some code. Your code looks like:
sub make_query {
my ($table, $columns, $conditions) = #_;
return "SELECT $columns FROM $table WHERE $conditions";
}
sub run_query {
my ($query) = #_;
$dbh->prepare($query);
...
}
run_query( make_query( 'foo', '*', '1=1' ) );
This doesn't let you do what you want to do. So you should structure
your program to do something like:
sub make_query {
my ($table, $columns, $conditions) = #_;
return +{
query => "SELECT $columns FROM $table WHERE $conditions",
table => $table,
} # an object might not be a bad idea
}
sub run_query {
my ($query) = #_;
$dbh->prepare($query->{query});
log_to_file( $query->{table}.'.log', ... );
...
}
run_query( make_query( 'foo', '*', '1=1' ) );
The API is the same, but now you have the information you need to log
the way you want.
Also, consider SQL::Abstract for dynamic SQL generation. My code
above is just an example.
Edit: OK, so you say you're using SQLite. It has an EXPLAIN command
which you could parse the output of:
sqlite> explain select * from test;
0|Trace|0|0|0|explain select * from test;|00|
1|Goto|0|11|0||00|
2|SetNumColumns|0|2|0||00|
3|OpenRead|0|2|0||00|
4|Rewind|0|9|0||00|
5|Column|0|0|1||00|
6|Column|0|1|2||00|
7|ResultRow|1|2|0||00|
8|Next|0|5|0||00|
9|Close|0|0|0||00|
10|Halt|0|0|0||00|
11|Transaction|0|0|0||00|
12|VerifyCookie|0|1|0||00|
13|TableLock|0|2|0|test|00|
14|Goto|0|2|0||00|
Looks like TableLock is what you would want to look for. YMMV, this
is a bad idea.
In general, in SQL, you cannot reliably deduce table names from result set, both for theoretical reasons (the result set may only consist of computed columns) and practical (the result set never includes table names - only column names - in its data).
So the only way to figure out the tables used is to stored them with (or deduce them from) the original query.
I've heard good things about the parsing ability of SQL::Statement but never used it before now myself.
use SQL::Statement;
use strict;
use warnings;
my $sql = <<" END_SQL";
SELECT TADA.fileName, TADA.labelName
FROM TADA
END_SQL
my $parser = SQL::Parser->new();
$parser->{RaiseError} = 1;
$parser->{PrintError} = 0;
my $stmt = eval { SQL::Statement->new($sql, $parser) }
or die "parse error: $#";
print join',',map{$_->name}$stmt->tables;