For the past couple of hours I've been trying to format a MySQL timestamp using DATE_FORMAT and it doesn't do anything!
Perl Code:
use CGI;
use DBI;
my $q = new CGI;
# Database connection goes here
my $sth_select = $dbh->prepare(
"SELECT DATE_FORMAT(timestamp, '%m/%d/%y') FROM foo"
);
$sth_select->execute() || die "Unable to execute query: $dbh->errstr";
if (my $ref = $sth_select->fetchrow_hashref()) {
print $q->header;
print " TIME: $ref->{timestamp}";
exit;
}
Results
TIME:
It doesn't print the formatted time at all, it is blank!
When I attempt to print the timestamp it doesn't print anything, but if I were to remove DATA_FORMAT and just simply do a SELECT timestamp FROM foo, then it prints the timestamp just fine, albeit not formatted though. Can somebody provide their insight on this matter, please?
The hash returned has as keys column headers as provided by the database. When using a function like that, the column header is actually "DATE_FORMAT(timestamp, '%m/%d/%y')".
Try modifying your SQL to be:
my $sth_select = $dbh->prepare("SELECT DATE_FORMAT(timestamp, '%m/%d/%y') AS timestamp FROM foo");
Related
got a problem with perl dbi sqlite.
I have set up a database (and checked it with sqlite command line).
Now i want to search in this database, which did not work.
So i tried to just do a 'SELECT *'
this prints only the first element in the database, but not as it should everything in this table.
I think the error that causes the select * to fail is the same that prevents me from using "like %..%" stuff.
This is the relevant code, if the code is correct and the database table seems good what else could have caused the problems ?
my $dbh = DBI->connect("dbi:SQLite:dbname=$dbfile","","") || die "Cannot connect: $DBI::errstr";
my $sth = $dbh->prepare('SELECT * FROM words');
$sth->execute;
my #result = $sth->fetchrow_array();
foreach( #result) {
print $_;
}
fetchrow_array() only fetches one row.
Try
while ( my #row = $sth->fetchrow_array ) {
print "#row\n";
}
According to the documentation, fetchrow_array
Fetches the next row of data and returns it as a list containing the field values.
If you want all of the data you can call fetchrow_array (or fetchrow_arrayref) repeatedly until you reach the end of the table, or you can use fetchall_arrayref:
The fetchall_arrayref method can be used to fetch all the data to be returned from a prepared and executed statement handle. It returns a reference to an array that contains one reference per row
The code would look like this
use strict;
use warnings;
use DBI;
my $dbfile = 'words.db';
my $dbh = DBI->connect("dbi:SQLite:dbname=$dbfile", '', '') or die "Cannot connect: $DBI::errstr";
my $sth = $dbh->prepare('SELECT * FROM words');
$sth->execute;
my $result = $sth->fetchall_arrayref;
foreach my $row ( #$result ) {
print "#$row\n";
}
I have been trying to use sql NOW() function while I update a table. But nothing happens to that date field, I think DBI just ignores that value. Code is :
dbUpdate($id, (notes => 'This was an update', filesize => '100.505', dateEnd => 'NOW()'));
and the function is :
sub dbUpdate {
my $id = shift;
my %val_hash = #_;
my $table = 'backupjobs';
my #fields = keys %val_hash;
my #values = values %val_hash;
my $update_stmt = "UPDATE $table SET ";
my $count = 1;
foreach ( #fields ) {
$update_stmt .= "$_ = ? ";
$update_stmt .= ', ' if ($count != scalar #fields);
$count++;
}
$update_stmt .= " WHERE ID = ?";
print "update_stmt is : $update_stmt\n";
my $dbo = dbConnect();
my $sth = $dbo->prepare( $update_stmt );
$sth->execute( #values, $id ) or die "DB Update failed : $!\n";
$dbo->disconnect || die "Failed to disconnect\n";
return 1;
}#dbUpdate
As you can see this is a "dynamic" generation of the sql query and hence I dont know where an sql date function(like now()) come.
In the given example the sql query will be
UPDATE backupjobs SET filesize = ? , notes = ? , dateEnd = ? WHERE ID = ?
with param values
100.55, This was an update, NOW(), 7
But the date column still shows 0000-00-........
Any ideas how to fix this ?
I have been trying to use sql NOW() function while I update a table. But
nothing happens to that date field, I think DBI just ignores that
value.
No, it doesn't. But the DB thinks you are supplying a datetime or timestamp data type when you are in fact trying to add the string NOW(). That of course doesn't work. Unfortunately DBI cannot find that out for you. All it does is change ? into an escaped (via $dbh->quote()) version of the string it got.
There are several things you could do:
Supply a current timestamp string in the appropriate format instead of the string NOW().
If you have it available, you can use DateTime::Format::MySQL like this:
use DateTime::Format::MySQL;
dbUpdate(
$id,
(
notes => 'This was an update',
filesize => '100.505',
dateEnd => DateTime::Format::MySQL->format_datetime(DateTime->now),
)
);
If not, just use DateTime or Time::Piece.
use DateTime;
my $now = DateTime->now->datetime;
$now =~ y/T/ /;
dbUpdate(
$id,
(notes => 'This was an update', filesize => '100.505', dateEnd => $now));
Tell your dbUpdate function to look for things like NOW() and react accordingly.
You can do something like this. But there are better ways to code this. You should also consider that there is e.g. CURRENT_TIMESTAMP() as well.
sub dbUpdate {
my $id = shift;
my %val_hash = #_;
my $table = 'backupjobs';
my $update_stmt = "UPDATE $table SET ";
# Check for the NOW() value
# This could be done with others too
foreach ( keys %val_hash ) {
if ($val_hash{$_} =~ m/^NOW\(\)$/i) {
$update_stmt .= "$_ = NOW()";
$update_stmt .= ', ' if scalar keys %val_hash > 1;
delete $val_hash{$_};
}
}
# Put everything together, join keeps track of the trailing comma
$update_stmt .= join(', ', map { "$_=?" } keys %val_hash );
$update_stmt .= " WHERE ID = ?";
say "update_stmt is : $update_stmt";
say "values are: ", join(', ', values %val_hash);
my $dbo = dbConnect();
my $sth = $dbo->prepare( $update_stmt );
$sth->execute( values %val_hash, $id ) or die "DB Update failed : $!\n";
$dbo->disconnect || die "Failed to disconnect\n";
return 1;
}
Write your queries yourself.
You're probably not going to do it and I'll not add an example since you know how to do it anyway.
Here's something else: Is this the only thing you do with your database while your program runs? It is not wise to connect and disconnect the database every time you make a query. It would be better for performance to connect the database once you need it (or at the beginning of the program, if you always use it) and just use this dbh/dbo everywhere. It saves a lot of time (and code).
Sorry but you can't use NOW() as an interpolated value like that.
when a ? is used in an SQL statement, the corresponding value is (via some mechanism or other) passed to the database escaped, so whatever value is interpreted as a string, not as SQL. So you are in effect attempting to use the string 'NOW()' as a date value, not the function NOW().
To use NOW() as a function, you will have to insert it into the SQL itself rather than pass it as a bound value. This means you will either have to use a hack of your dbupdate function or write a new one, or obtain the time as a string in perl and pass the resulting string the dbupdate.
I am writing a Perl script that is using the DBI module and is connecting to a Sybase DB. I am calling a stored procedure (one that I don't have access to so I cannot post sample code) and when I get data back I get an error that reads "error_handler: Data-conversion resulted in overflow". I still get data back and after doing some intensive research it seems that some data types in the columns (such as BigInt, nvarchar, etc) are the culprits. Now the question is, how can I fix this? Can this be fixed on the client side or can it only be fixed on the server side?
my $dbh = DBI->connect("DBI:Sybase:server=$server", $username, $password, {PrintError => 0}) or die;
$dbh->do("use $database") or die;
my $sql = &getQuery;
my $sth = $dbh->prepare($sql) or die;
$sth->execute() or die;
while ($rowRef = $sth->fetchrow_arrayref) #Error seems to occur here
{
#Parse through each row
}
Part of the FreeTDS 0.82 log that explains the problem:
_ct_bind_data(): column 7 is type 38 and has length 8
_ct_get_server_type(0)
_ct_get_client_type(type 38, user 0, size 8)
cs_convert(0x18dfed40, 0x7fff73216050, 0x18e44250, 0x7fff73215fa0, 0x18e387c0, 0x18e45a64)
_ct_get_server_type(30)
_ct_get_server_type(0)
converting type 127 (8 bytes) to type = 47 (9 bytes)
cs_convert() calling tds_convert
cs_convert() tds_convert returned 10
cs_prretcode(0)
cs_convert() returning CS_FAIL
cs_convert-result = 1
The problem is on the FreeTDS side. I've had the same problem before and successfully fixed it by converting the returned fields to varchar in the select statement.
Given you don't have access to modify the original query, you can do some regex search and replace on the returned $sql variable in your code. In particular, if the original query has a part that looks like
SELECT field1, field2, field3 FROM ...
After you retrieve the query statement, you may run
my $new_sql;
if ($sql =~ /SELECT\s+(.*)\s+FROM/i) { # match selected field string
my $field_str = $1;
my #fields = split ",", $field_str; # parse individual fields
map s/\s//g, #fields; # get rid of spaces
my $new_str = join ", ", (map {sprintf "convert(varchar, $_)"} #fields); # construct new query string
my $quoted_field_str = quotemeta($field_str); # prepare regex replacement string
$new_sql = $sql;
$new_sql =~ s/$quoted_field_str/$new_str/i # actual replacement
}
print $new_sql;
Of course, if your original statement is more complex, you should print it out and check how to modify it with a generic replacement bearing the same spirit. Alternatively, you can ask your DBA (or whoever has access to the stored procedure) to modify the actual query directly.
Hope this helps.
Have some Perl code which is using the DBI module - (the code is at work, I can post it in the morning if needed) - but mainly trying to get a sense of what DBI needs to do an update to a row -- and get either errors back, or confirmation that the UPDATE was executed.
(Below is just a basic example, feel free to give your own example and sample DDL if you want... just want some code that I know works. I've run my code via the Perl PtkDB debugger, and can "see" the SQL it generating and executing -- even paste in in the MySQL consol and execute it... but it's doing nothing in the Perl, even thought the select statements are working. Mainly just want a better idea of how DBI is handling UPDATE to MySQL, and if there's any built in feature in DBI that would make debugging this more simple. Thanks!)
So, please supply one full Perl script that:
Sets the connection (MySQL)
SELECT row two based on ID and get the first and last name
Lowercase the names
UPDATE the table
disconnect
Sample TABLE
<COL01>Id <COL02>FirstName <COL03>LastName
<ROW01-COL01>1 <ROW01-COL02>John <ROW01-COL03>Smith
<ROW02-COL01>2 <ROW02-COL02>Jane <ROW02-COL03>Doe
UPDATE (1): Code in question is below. The ONLY thing I've changed is remove code not related to the issue and the config info (eg database name, user, password, etc.) and made the value production for the variables super simple. This code was created by someone else and in a legacy code base.
use strict;
use warnings;
use DBI;
sub dbOpen {
my $dsn;
my $dbh;
$dsn = "DBI:mysql:database=databasename;host=localhost;port=3306";
$dbh = DBI->connect( $dsn, "root", "password" ) ||
print STDERR "FATAL: Could not connect to database.\n$DBI::errstr\n";
$dbh->{ AutoCommit } = 0;
return($dbh);
} # END sub dbOpen
my $Data;
$Data = &dbOpen();
my ($sql,$rs,$sql_update_result);
my $column2,
my $column3;
my $id;
$column2 = 2,
$column3 = 3;
$id = 1;
$sql = "UPDATE table SET column1 = NULL, column2 = ".$column2.", column3 = ".$column3." WHERE id = ".$id.";";
$rs = $Data->prepare( $sql );
$rs->execute() || &die_clean("Couldn't execute\n$sql\n".$Data->errstr."\n" );
($sql_update_result) = $rs->fetchrow;
$Data->disconnect();
DDL for MySQL -- if needed, just comment and I'll post one.
UPDATE (2):
Final found one complete example, though it's only for a select statement and not even inserting any VARs into the SQL: http://search.cpan.org/~timb/DBI/DBI.pm#Simple_Examples
Almost copy and paste from DBI Synopsis:
use DBI;
$dbh = DBI->connect($data_source, $username, $auth, \%attr);
$statement = "UPDATE some_table SET som_col = ? WHERE id = ?";
$rv = $dbh->do($statement, undef, $som_val, $id);
$DBI::err && die $DBI::errstr;
$rc = $dbh->disconnect;
I prefer to use do when updating or deleting since these operations doesn't return any row.
So, in order to have a little debug, i would modify your code like this:
my $sql = "UPDATE table SET column1=NULL, column2=$column2, column3=$column3 WHERE id=$id";
print STDERR "SQL: $sql\n"
my $numrows = $Data->do($sql);
if (not defined $numrows) {
print STDERR "ERROR: $DBI::errstr";
} else {
print STDERR "INFO: $numrows rows updated";
}
You can measure query response times from within your perl code, but since it is a database thing, i recommend you using any Mysql specialized tool (i don't use MySQL, sorry).
Have you considered something a bit higher level - like DBIx::Class?
You don't need to return the values, lowercase them in Perl, then update the rows. Just do that in one SQL statement:
my $sql = "UPDATE table SET column2=lower(column2) WHERE id = ?";
$sth = $dbh->prepare($sql);
foreach my $id (#ids) {
$sth->execute($id);
}
You also want to use placeholders to prevent Bobby Tables from visiting.
I am writing small snippets in Perl and DBI (SQLite yay!)
I would like to log some specific queries to text files having the same filename as that of the table name(s) on which the query is run.
Here is the code I use to dump results to a text file :
sub dumpResultsToFile {
my ( $query ) = #_;
# Prepare and execute the query
my $sth = $dbh->prepare( $query );
$sth->execute();
# Open the output file
open FILE, ">results.txt" or die "Can't open results output file: $!";
# Dump the formatted results to the file
$sth->dump_results( 80, "\n", ", ", \*FILE );
# Close the output file
close FILE or die "Error closing result file: $!\n";
}
Here is how I can call this :
dumpResultsToFile ( <<" END_SQL" );
SELECT TADA.fileName, TADA.labelName
FROM TADA
END_SQL
What I effectively want is, instead of stuff going to "results.txt" ( that is hardcoded above ), it should now go to "TADA.txt".
Had this been a join between tables "HAI" and "LOL", then the resultset should be written to "HAI.LOL.txt"
Is what I am saying even possible using some magic in DBI?
I would rather do without parsing the SQL query for tables, but if there is a widely used and debugged function to grab source table names in a SQL query, that would work for me too.
What I want is just to have a filename
that gives some hint as to what query
output it holds. Seggregating based on
table name seems a nice way for now.
Probably not. Your SQL generation code takes the wrong approach. You are hiding too much information from your program. At some point, your program knows which table to select from. Instead of throwing that information away and embedding it inside an opaque SQL command, you should keep it around. Then your logger function doesn't have to guess where the log data should go; it knows.
Maybe this is clearer with some code. Your code looks like:
sub make_query {
my ($table, $columns, $conditions) = #_;
return "SELECT $columns FROM $table WHERE $conditions";
}
sub run_query {
my ($query) = #_;
$dbh->prepare($query);
...
}
run_query( make_query( 'foo', '*', '1=1' ) );
This doesn't let you do what you want to do. So you should structure
your program to do something like:
sub make_query {
my ($table, $columns, $conditions) = #_;
return +{
query => "SELECT $columns FROM $table WHERE $conditions",
table => $table,
} # an object might not be a bad idea
}
sub run_query {
my ($query) = #_;
$dbh->prepare($query->{query});
log_to_file( $query->{table}.'.log', ... );
...
}
run_query( make_query( 'foo', '*', '1=1' ) );
The API is the same, but now you have the information you need to log
the way you want.
Also, consider SQL::Abstract for dynamic SQL generation. My code
above is just an example.
Edit: OK, so you say you're using SQLite. It has an EXPLAIN command
which you could parse the output of:
sqlite> explain select * from test;
0|Trace|0|0|0|explain select * from test;|00|
1|Goto|0|11|0||00|
2|SetNumColumns|0|2|0||00|
3|OpenRead|0|2|0||00|
4|Rewind|0|9|0||00|
5|Column|0|0|1||00|
6|Column|0|1|2||00|
7|ResultRow|1|2|0||00|
8|Next|0|5|0||00|
9|Close|0|0|0||00|
10|Halt|0|0|0||00|
11|Transaction|0|0|0||00|
12|VerifyCookie|0|1|0||00|
13|TableLock|0|2|0|test|00|
14|Goto|0|2|0||00|
Looks like TableLock is what you would want to look for. YMMV, this
is a bad idea.
In general, in SQL, you cannot reliably deduce table names from result set, both for theoretical reasons (the result set may only consist of computed columns) and practical (the result set never includes table names - only column names - in its data).
So the only way to figure out the tables used is to stored them with (or deduce them from) the original query.
I've heard good things about the parsing ability of SQL::Statement but never used it before now myself.
use SQL::Statement;
use strict;
use warnings;
my $sql = <<" END_SQL";
SELECT TADA.fileName, TADA.labelName
FROM TADA
END_SQL
my $parser = SQL::Parser->new();
$parser->{RaiseError} = 1;
$parser->{PrintError} = 0;
my $stmt = eval { SQL::Statement->new($sql, $parser) }
or die "parse error: $#";
print join',',map{$_->name}$stmt->tables;