When I use the same request in mySQL Workbench, the count of results is 41100.
In my script, the count is 1015. I have no error in my code (I use die for see it).
What is the cause of this difference?
# Create the statement
# select all id in table contacts
$statement = "select distinct contacts.id from $database.contacts";
# Prepare and execute the SQL query
$sqlQuery = $connectGrc->prepare($statement)
# Execute the statement
$sqlQuery->execute()
# Make id in array
while($oneId = $sqlQuery->fetchrow_array())
{
push(#ArrayId,$oneId);
}
There's nothing obviously wrong (except for the lack of error checking and that your comments are not program comments). Try connecting with RaiseError on, 'use strict;', and see if there are problems which you otherwise might not see. Also, if all you want is a one-column array of results, and can settle for an array reference, consider selectcol_arrayref() for convenience.
Related
A line of code like
my $sql_query = "SELECT * FROM Users WHERE user='$user';";
might introduce an SQL injection vulnerability into your program. To avoid this one could use something like
my $sth = $dbh->prepare("SELECT * FROM Users WHERE user='?';");
$dbh->execute($user);
However, in the code I am currently working on the following is used
$sql_query = "SELECT * FROM Users WHERE user='" . $user . "';";
$dbh->prepare($sql_query);
$dbh->execute();
Does this actually work? If yes, are there any differences to what I would have done? What are the advantages and disadvantages?
my $sth = $dbh->prepare("SELECT * FROM Users WHERE user='?'");
This won't work because it's searching for a literal '?' character — not a parameter. If you try to send a value for the parameter, MySQL will be like, "what do you want me to do with this?" because the query has no parameter placeholder.
If you want to use a parameter, you must NOT put the parameter placeholder inside string delimiters in the SQL query, even if the parameter will take string or datetime value:
my $sth = $dbh->prepare("SELECT * FROM Users WHERE user=?");
The next example:
$sql_query = "SELECT * FROM Users WHERE user='" . $user . "'";
$dbh->prepare($sql_query);
$dbh->execute();
That will run the query, but it's NOT safe. You can prepare any query even if it has no parameters.
Using prepare() is not what makes queries safe from SQL injection. What makes it safer is using parameters to combine dynamic values instead of doing string-concatenation like you're doing in this example.
But using parameters does require the use of prepare().
PS: You don't need to put ; at the end of your SQL queries when you run them one at a time programmatically. The separator is only needed if you run multiple queries, like in an SQL script, or in a stored procedure. In your examples, the ; is harmless but MySQL doesn't require it, and it will just ignore it.
You have the following table called Pets:
name age pet
------------------------
Carol 25 null
Stean 23 cat
Mel 24 dog
Rich 24 rabbit
In a MySQL database on the server mydbserver with the user of 'user' with a
password of 'password'.
Do the following:
1) Create a Class::DBI connection to this database with the above credentials ( DBI.pm ).
2) Create a Class for the table Pets ( Pet.pm )
3) Create a program that prints all the names of people in the Pets table and what kind (if any )
of pet he/she has sorted by name then age.
Here is the code I wrote.....
#!/usr/bin/perl
package Pet::DBI;
use DBI;
use strict;
use base 'Class::DBI';
Pet::DBI->set_db('Main','dbi:mysql:dname', 'user', 'password')
or die $DBI::errstr "\n";
1;
package Pet::Pets;
use base 'Pet::DBI';
use strict;
use warning;
Pet::Pets->table('Pets');
Pet::Pets->columns(All => qw/name age pet/);
1;
use Pet::Pets;
my #pets = Pet::Pets->retrieve_all;
for (sort {$a->name cmp $b->name} || {$a->age <=> $b->age} #Pets) {
print "Name:".$_->name ' => '."Age". $_->age"\n";
}
1;
It's basically correct, but there's a number of small problems.
It's not necessary to load DBI, Class::DBI will take care of that for you.
You should be using connection instead of set_db("Main", ...). set_db comes from Ima::DBI and it's not polite (or necessary) to peek under the hood like that.
Although this isn't directly documented in Class::DBI (it should be), its inherited from Ima::DBI, there's no need to check for DBI errors. RaiseError is on and if the connection fails it will throw an error.
You have a typo, use warning; instead of use warnings;.
Unless you have stitched three files together for the post, if the code is all in one file the 1; statements do nothing. use Pet::Pets will not work because there is no Pet/Pets.pm file. You don't have to use a class which is already in the same file.
In general, avoid using $_ if you don't have to, too many things can quietly use or change it. Instead, give the for loop a proper variable like for my $person.
sort only takes one block, but you're basically correct. It should be sort { ($a->name cmp $b->name) || ($a->age <=> $b->age) } #Pets
To avoid reading the whole, potentially very large, table into memory, the sorting should really be done in the database with an ORDER BY name ASC, age ASC and then retrieved a row at a time using an iterator. Unfortunately, retrieve_all does not support any options. You can use retrieve_from_sql to add arbitrary SQL to the end of the basic SELECT. my $all_people = Pet::Pets->retrieve_from_sql("ORDER BY name ASC, age ASC"); Then your data will already be sorted and can be read a row at a time. while( my $person = $all_people->next ) { ... }
You're missing a . in "Age". $_->age"\n".
Null values in a database come back as undef. You'll want to check if $_->pet is defined and if not use some other string like "no pet" or just a blank "".
You're printing the person's age, the question asks for their pet.
Otherwise, it should work.
But really, tell whomever gave you this homework to stop telling people to use Class::DBI. They can email me if they like, schwern#pobox.com.
It seems to me that there simply has to be a better way of doing this, but i still haven't found one. And i'm sure i'm not the only one who could use a way to do this: Run an SQL query that only produces one field in one row, then assign that field to a scalar. (In my case, if the query results in more than one field/row, then i have bigger things to worry about than the script breaking).
For example, to get the timestamp from the SQL server, one could use:
my $timestamp;
my $cmd = $dbh->prepare('SELECT cast(now() AS timestamp);') or die $!;
$cmd->execute();
while (my #asd = $cmd->fetchrow_array) { $timestamp = $asd[0] }
Dirty, but it works. But using 4 lines seem a bit much for a simple assignment, especially considering how well perl and postgresql can communicate with eachother via DBI. Sure, i could write a subroutine for it, but isn't there something native that allows me to fetch data as easily as i submit data with $dbh->do() ?
And yes, i did try google.
Usually I write:
$value = $dbh->selectall_arrayref($sql)->[0]->[0];
There's always selectrow_array:
selectrow_array
#row_ary = $dbh->selectrow_array($statement);
#row_ary = $dbh->selectrow_array($statement, \%attr);
#row_ary = $dbh->selectrow_array($statement, \%attr, #bind_values);
This utility method combines prepare, execute and fetchrow_array into a single call.
So something like this:
my $timestamp = $dbh->selectrow_array('select cast(now() as timestamp)');
There's also selectrow_arrayref and selectrow_hashref for similar situations.
From perldoc DBI:
"selectrow_arrayref"
$ary_ref = $dbh->selectrow_arrayref($statement);
$ary_ref = $dbh->selectrow_arrayref($statement, \%attr);
$ary_ref = $dbh->selectrow_arrayref($statement, \%attr, #bind_values);
This utility method combines "prepare", "execute" and
"fetchrow_arrayref" into a single call. It returns the first row of
data from the statement. The $statement parameter can be a previously
prepared statement handle, in which case the "prepare" is skipped.
If any method fails, and "RaiseError" is not set, "selectrow_array"
will return undef.
That will get you most of the way. You still need to do some error checking, but you would be doing that anyway.
Wouldn't fetchrow_array actually only return a scalar as you're only asking for one column?
So I started to get familiar with Perl and I wrote my first Db script.
Now I am trying to select data from atable which is huge and trying to insert into a summary table based on some criteria.
Now there are chances , that select query may fail or the insert query may fail due to timeout or other database issues that is beyond my control.
Eventually my script is going to be cron script.
Can I log just the errors that i encounter for the connection,inserts and selects into a file generated in the script?
$logfile = $path.'logs/$currdate.log';
here is my code:
my $SQL_handled="SELECT division_id,region_id, NVL(COUNT(*),0) FROM super_tab GROUP BY division_id,region_id;";
my $result_handled = $dbh->prepare($SQL_handled);
$result_handled->execute();
while (my ($division_id,$region_id,$count ) = $result_handled->fetchrow_array()){
my $InsertHandled="INSERT INTO summary_tab (date_hour, division_id, region_id,volume) VALUES ('$current',$division_id,$region_id,$market_id,'$service_type','$handled',$count);";
my $result_insert_handled = $dbh->prepare($InsertHandled);
$result_insert_handled->execute();
}
something like
if(DBI-query failed ) {
// log the error onto the above logpath
}
Its usually done like this
my $SQL_handled="SELECT division_id,region_id, NVL(COUNT(*),0) FROM super_tab GROUP BY division_id,region_id;";
my $result_handled = $dbh->prepare($SQL_handled);
my $retval = $result_handled->execute();
if(!$retval){
#open a log file and write errors
writelog();
die "Error executing SQL SELECT - $dbh->errstr";
}
while(my ($division_id,$region_id,$count ) = $result_handled->fetchrow_array()){....
}
---------------------------------
sub writelog{
my $path = "/path/to/logfile";
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
$year += 1900;
$mon++;
my $currdate = "$mon$mday$year";
$logfile = $path . "/$currdate.log";
open (OUT, ">>$logfile");
print OUT "There was an error encountered while executing SQL- $dbh->errstr \n";
close(OUT);
}
You can also use $dbh->err; which returns the native Oracle error code to trap the error and exit accordingly.
The above, basic exception handling can be performed for every execute() method call in your script. Remember, DBI will have AutoCommit set to 1 (enabled) by default, unless explicitly disabled. So your transactions would be auto committed per insert, in order to handle the ATOMICITY of the entire transaction, you can disable autocommit and use $dbh->commit and $dbh->rollback to handle when you want to commit, or may be use some custom commit point (for larger sets of data).
Or the below can be used while connecting to the DB
$dbh = DBI->connect( "dbi:Oracle:abcdef", "username", "password" , {
PrintError => 0, ### Don't report errors via warn( )
RaiseError => 1 ### Do report errors via die( )
} );
this would automatically report all errors via die. The RaiseError is usually turned off by default.
Also if I understand you correctly then, by cron you mean you would be calling it from a shell cron job. In that case, call to your perl script from the cron itself can be redirected to log files something like below
perl your_perl.pl >> out.log 2>> err.log
out.log will contain regular logs and err.log will contain errors (specifically thrown by DBI prepare() or execute() methods too). In this case, you also need to make sure you use proper verbiage in print or die so that the logs look meaningful.
First, bear in mind that if you put an email address at the top of your crontab file any output from the cron job will be emailed to you:
MAILTO=me#mydomain.com
Second, if you set DBI's RaiseError to 1 when you connect you do not need to check every call, DBI will raise an error whenever one happens.
Third, DBI has an error handler callback. You register a handler and it is called whenever an error occurs with the handle in error and error text etc. If you return false from the error handler, DBI works as it would without the handler and goes on to die or warn. As a result, it is easier to set RaiseError and create an error handler than as Annjawn suggested.
Lastly, if you don't want to do this yourself, you can use something like DBIx::Log4perl and simply ask for it to log errors and nothing else. Any errors will be written to your Log4perl file and they include the SQL being executed, parameters etc.
How do I using a variable for the name of a table in a DBI query? I know how to use placeholders as part of the where clause, but how do I do this for the table name?
I would like to do something like this:
my $table_name='table1';
my $query = $dbh_cgi->prepare("select * from ?");
$query->execute($table_name);
So far, I end up getting a MySQL syntax error because DBI adds quotes around the name, table1.
One of the limitations of placeholders is that they can't be used for table names. Instead of using a placeholder, you can use a variable. To make sure the variable's contents are safe, use quote_identifier(), for example:
my $table_name='table1'; #-- Or get this from somewhere else
my $safe_table_name = $dbh_cgi->quote_identifier($table_name);
my $query = $dbh_cgi->prepare("select * from $safe_table_name");
$query->execute();