When I fetch the data this way is it possible then to access the column names and the column types or do I need an explicit prepare to reach this?
use DBI;
my $dbh = DBI->connect( ... );
my $select = "...";
my #arguments = ( ... );
my $ref = $dbh->selectall_arrayref( $select, {}, #arguments, );
Update:
With prepare I would do it this way:
my $sth = $dbh->prepare( $select );
$sth->execute( #arguments );
my $col_names = $sth->{NAME};
my $col_types = $sth->{TYPE};
my $ref = $sth->fetchall_arrayref;
unshift #$ref, $col_names;
The best solution is to use prepare to get a statement handle, as you describe in the second part of your question. If you use selectall_hashref or selectall_arrayref, you don't get a statement handle, and have to query the column type information yourself via $dbh->column_info (docs):
my $sth = $dbh->column_info('','',$table,$column); # or $column='' for all
my $info = $sth->fetchall_arrayref({});
use Data::Dumper; print Dumper($info);
(specifically, the COLUMN_NAME and TYPE_NAME attributes).
However, this introduces a race condition if the table changes schema between the two queries.
Also, you may use selectall_arrayref with the Slice parameter to fetch all the columns into a hash ref, it needs no prepared statement and will return an array ref of the result set rows, with each rows columns the key's to a hash and the values are the column values. ie:
my $result = $dbh->selectall_arrayref( qq{
SELECT * FROM table WHERE condition = value
}, { Slice => {} }) or die "Error: ".$dbh->errstr;
$result = [
[0] = { column1 => 'column1Value', column2 => 'column2Value', etc...},
[1] = { column1 => 'column1Value', column2 => 'column2Value', etc...},
];
Making it easy to iterate over results.. ie:
for my $row ( #$results ){
print "$row->{column1Value}, $row->{column2Value}\n";
}
You can also specify which columns to extract but it's pretty useless due to the fact it's more efficient to do that in your SQL query syntax.
{ Slice => { column1Name => 1, column2Name => 1 } }
That would only return the values for column1Name and column2Name just like saying in your SQL:
SELECT column1Name, column2Name FROM table...
Related
sub insert {
my ($self) = #_;
my $data = $self->{data};
my $keys = join( ', ', keys $data);
my $values = join( ', ', map qq('$_'), values $data);
my $sql = "INSERT INTO tbl ($keys) VALUES ($values);";
my $sth = $self->{dbh}->prepare($sql);
$sth->execute();
}
I have a method that inserts the contents of a hash ref into a one table sqlite database. I was wondering if there was a simple way to add a column to the table if the hash key is not already a column. Obviously if one of the keys is not a column name, the insert will fail. Can I capitalize on that failure, add the missing columns, and redo the insert. Or would I have to check all columns against all the keys each time I want to insert into the database? (All keys have TEXT values)
Alter your table based on the info you get using PRAGMA
my $inf_query = $db->prepare("PRAGMA table_info('tbl')");
$inf_query->execute();
my #inf = map { $_->[1] } #{$inf_query->fetchall_arrayref()};
#inf will be an array containing the columns present in the table, and you can use that info to construct your ALTER query.
Edited to return an array you can use to grep ;)
I have been trying to use sql NOW() function while I update a table. But nothing happens to that date field, I think DBI just ignores that value. Code is :
dbUpdate($id, (notes => 'This was an update', filesize => '100.505', dateEnd => 'NOW()'));
and the function is :
sub dbUpdate {
my $id = shift;
my %val_hash = #_;
my $table = 'backupjobs';
my #fields = keys %val_hash;
my #values = values %val_hash;
my $update_stmt = "UPDATE $table SET ";
my $count = 1;
foreach ( #fields ) {
$update_stmt .= "$_ = ? ";
$update_stmt .= ', ' if ($count != scalar #fields);
$count++;
}
$update_stmt .= " WHERE ID = ?";
print "update_stmt is : $update_stmt\n";
my $dbo = dbConnect();
my $sth = $dbo->prepare( $update_stmt );
$sth->execute( #values, $id ) or die "DB Update failed : $!\n";
$dbo->disconnect || die "Failed to disconnect\n";
return 1;
}#dbUpdate
As you can see this is a "dynamic" generation of the sql query and hence I dont know where an sql date function(like now()) come.
In the given example the sql query will be
UPDATE backupjobs SET filesize = ? , notes = ? , dateEnd = ? WHERE ID = ?
with param values
100.55, This was an update, NOW(), 7
But the date column still shows 0000-00-........
Any ideas how to fix this ?
I have been trying to use sql NOW() function while I update a table. But
nothing happens to that date field, I think DBI just ignores that
value.
No, it doesn't. But the DB thinks you are supplying a datetime or timestamp data type when you are in fact trying to add the string NOW(). That of course doesn't work. Unfortunately DBI cannot find that out for you. All it does is change ? into an escaped (via $dbh->quote()) version of the string it got.
There are several things you could do:
Supply a current timestamp string in the appropriate format instead of the string NOW().
If you have it available, you can use DateTime::Format::MySQL like this:
use DateTime::Format::MySQL;
dbUpdate(
$id,
(
notes => 'This was an update',
filesize => '100.505',
dateEnd => DateTime::Format::MySQL->format_datetime(DateTime->now),
)
);
If not, just use DateTime or Time::Piece.
use DateTime;
my $now = DateTime->now->datetime;
$now =~ y/T/ /;
dbUpdate(
$id,
(notes => 'This was an update', filesize => '100.505', dateEnd => $now));
Tell your dbUpdate function to look for things like NOW() and react accordingly.
You can do something like this. But there are better ways to code this. You should also consider that there is e.g. CURRENT_TIMESTAMP() as well.
sub dbUpdate {
my $id = shift;
my %val_hash = #_;
my $table = 'backupjobs';
my $update_stmt = "UPDATE $table SET ";
# Check for the NOW() value
# This could be done with others too
foreach ( keys %val_hash ) {
if ($val_hash{$_} =~ m/^NOW\(\)$/i) {
$update_stmt .= "$_ = NOW()";
$update_stmt .= ', ' if scalar keys %val_hash > 1;
delete $val_hash{$_};
}
}
# Put everything together, join keeps track of the trailing comma
$update_stmt .= join(', ', map { "$_=?" } keys %val_hash );
$update_stmt .= " WHERE ID = ?";
say "update_stmt is : $update_stmt";
say "values are: ", join(', ', values %val_hash);
my $dbo = dbConnect();
my $sth = $dbo->prepare( $update_stmt );
$sth->execute( values %val_hash, $id ) or die "DB Update failed : $!\n";
$dbo->disconnect || die "Failed to disconnect\n";
return 1;
}
Write your queries yourself.
You're probably not going to do it and I'll not add an example since you know how to do it anyway.
Here's something else: Is this the only thing you do with your database while your program runs? It is not wise to connect and disconnect the database every time you make a query. It would be better for performance to connect the database once you need it (or at the beginning of the program, if you always use it) and just use this dbh/dbo everywhere. It saves a lot of time (and code).
Sorry but you can't use NOW() as an interpolated value like that.
when a ? is used in an SQL statement, the corresponding value is (via some mechanism or other) passed to the database escaped, so whatever value is interpreted as a string, not as SQL. So you are in effect attempting to use the string 'NOW()' as a date value, not the function NOW().
To use NOW() as a function, you will have to insert it into the SQL itself rather than pass it as a bound value. This means you will either have to use a hack of your dbupdate function or write a new one, or obtain the time as a string in perl and pass the resulting string the dbupdate.
I want to copy a single record from a table, modify some of the fields in the record and insert into the same table. The table has 90 columns.
Thought of using insert..select in one statement but there are 90 columns and i need to tell the column name in the select query. How can i do it in a better way in perldbi. Pls provide me an example.
Fetch and cache the column names for your table using the NAME statement attribute
my $sth = $dbh->prepare('SELECT * FROM my_table where 1=0');
$sth->execute;
my $cols = $sth->{NAME};
then use $cols to construct your insert...select using some replacement function to inject your modifications in the select.
my %mods_for_column = ( 'foo' => 'foo+10', 'bar' => 'trim(bar)' );
my $inscols = join(',', #$cols);
my $selcols = join(',',
map { exists($mods_for_column($_)) ? $mods_for_column($_) : $_ } #$cols
);
my $sql = "insert into my_table ($inscols) select $selcols from my_table where mumble...";
In this script I have problems with file-name-extensions:
if I use /home/mm/test_x it works, with file named /home/mm/test_x.csv it doesn't:
#!/usr/bin/env perl
use warnings; use strict;
use 5.012;
use DBI;
my $table_1 = '/home/mm/test_1.csv';
my $table_2 = '/home/mm/test_2.csv';
#$table_1 = '/home/mm/test_1';
#$table_2 = '/home/mm/test_2';
my $dbh = DBI->connect( "DBI:CSV:" );
$dbh->{RaiseError} = 1;
$table_1 = $dbh->quote_identifier( $table_1 );
$table_2 = $dbh->quote_identifier( $table_2 );
my $sth = $dbh->prepare( "SELECT a.id, a.name, b.city FROM $table_1 AS a NATURAL JOIN $table_2 AS b" );
$sth->execute;
$sth->dump_results;
$dbh->disconnect;
Output with file-name-extention:
DBD::CSV::st execute failed:
Execution ERROR: No such column '"/home/mm/test_1.csv".id' called from /usr/local/lib/perl5/site_perl/5.12.0/x86_64-linux/DBD/File.pm at 570.
Output without file-name-extension:
'1', 'Brown', 'Laramie'
'2', 'Smith', 'Watertown'
2 rows
Is this a bug?
cat test_1.csv
id,name
1,Brown
2,Smith
5,Green
cat test_2.csv
id,city
1,Laramie
2,Watertown
8,Springville
DBD::CSV provides a way to map the table names you use in your queries to filenames. The same mechanism is used to set up per-file attributes like line ending, field separator etc. look for 'csv_tables' in the DBD::CSV documentation.
#!/usr/bin/env perl
use warnings;
use strict;
use DBI;
my $dbh = DBI->connect("DBI:CSV:f_dir=/home/mm", { RaiseError => 1 });
$dbh->{csv_tables}->{table_1} = {
'file' => 'test_1.csv',
'eol' => "\n",
};
$dbh->{csv_tables}->{table_2} = {
'file' => 'test_2.csv',
'eol' => "\n",
};
my $sth = $dbh->prepare( "SELECT a.id, a.name, b.city FROM table_1 AS a NATURAL JOIN table_2 AS b" );
$sth->execute();
$sth->dump_results();
$dbh->disconnect();
In my case I had to specify a line ending character, because I created the CSV files in vi so they ended up with Unix line endings whereas DBD::CSV assumes DOS/Windows line-endings regardless of the platform the script is run on.
I looks like even this works:
#!/usr/bin/env perl
use warnings; use strict;
use 5.012;
use DBI;
my $dbh = DBI->connect("DBI:CSV:f_dir=/home/mm/Dokumente", undef, undef, { RaiseError => 1, });
my $table = 'new.csv';
$dbh->do( "DROP TABLE IF EXISTS $table" );
$dbh->do( "CREATE TABLE $table ( id INT, name CHAR(64), city CHAR(64) )" );
my $sth_new = $dbh->prepare( "INSERT INTO $table( id, name, city ) VALUES ( ?, ?, ? )" );
$dbh->{csv_tables}->{table_1} = { 'file' => '/tmp/test_1.csv', 'eol' => "\n", };
$dbh->{csv_tables}->{table_2} = { 'file' => '/tmp/test_2.csv', 'eol' => "\n", };
my $sth_old = $dbh->prepare( "SELECT a.id, a.name, b.city FROM table_1 AS a NATURAL JOIN table_2 AS b" );
$sth_old->execute();
while ( my $hash_ref = $sth_old->fetchrow_hashref() ) {
state $count = 1;
$sth_new->execute( $count++, $hash_ref->{'a.name'}, $hash_ref->{'b.city'} );
}
$dbh->disconnect();
I think you might want to take a look at the f_ext and f_dir attributes. You can then class your table names as "test_1" and "test_2" without the csv but the files used will be test_1.csv and test_2.csv. The problem with a dot in the table name is a dot is usually used for separating the schema from the table name (see f_schema).
I need to insert values in database using Perl's DBI module. I have parsed a file to obtain these values and hence these values are present in an arrays, say #array1, #array2, #array3. I know how to insert one value at a time but not from an arrays.
I know insert one value at a time:
$dbh = DBI->connect("dbi:Sybase:server=$Srv;database=$Db", "$user", "$passwd") or die "could not connect to database";
$query= "INSERT INTO table1 (id, name, address) VALUES (DEFAULT, tom, Park_Road)";
$sth = $dbh->prepare($query) or die "could not prepare statement\n";
$sth-> execute or die "could not execute statement\n $command\n";
I am not sure if I have array1 containing ids, array2 containing names, and array3 containing address, how would I insert values.
Since you have parallel arrays, you could take advantange of execute_array:
my $sth = $dbh->prepare('INSERT INTO table1 (id, name, address) VALUES (?, ?, ?)');
my $num_tuples_executed = $sth->execute_array(
{ ArrayTupleStatus => \my #tuple_status },
\#ids,
\#names,
\#addresses,
);
Please note that this is a truncated (and slightly modified) example from the documentation. You'll definitely want to check out the rest of it if you decide to use this function.
Use placeholders.
Update: I just realized you have parallel arrays. That is really not a good way of working with data items that go together. With that caveat, you can use List::MoreUtils::each_array:
#!/usr/bin/perl
use strict; use warnings;
use DBI;
use List::MoreUtils qw( each_array );
my $dbh = DBI->connect(
"dbi:Sybase:server=$Srv;database=$Db",
$user, $passwd,
) or die sprintf 'Could not connect to database: %s', DBI->errstr;
my $sth = $dbh->prepare(
'INSERT INTO table1 (id, name, address) VALUES (?, ?, ?)'
) or die sprintf 'Could not prepare statement: %s', $dbh->errstr;
my #ids = qw( a b c);
my #names = qw( d e f );
my #addresses = qw( g h i);
my $it = each_array(#ids, #names, #address);
while ( my #data = $it->() ) {
$sth->execute( #data )
or die sprintf 'Could not execute statement: %s', $sth->errstr;
}
$dbh->commit
or die sprintf 'Could not commit updates: %s', $dbh->errstr;
$dbh->disconnect;
Note that the code is not tested.
You might also want to read the FAQ: What's wrong with always quoting "$vars"?.
Further, given that the only way you are handling error is by dying, you might want to consider specifying { RaiseError => 1 } in the connect call.
How could you not be sure what your arrays contain? Anyway the approach would be the iterate through one array and assuming the other arrays have corresponding values put those into the insert statement
Another way would be to use a hash as an intermediate storage area. IE:
my $hash = {};
foreach(#array1) {
$hash->{id} = $array1[$_];
$hash->{name} = $array2[$_];
$hash->{address} = $array3[$_];
}
foreach( keys %$hash ) {
$sql = "insert into table values(?,?,?)";
$sth = $dbh->prepare($sql) or die;
$sth->execute($hash->{id}, $hash->{name}, $hash->{address}) or die;
}
Though again this depends on the three arrays being synced up. However you could modify this to do value modifications or checks or greps in the other arrays within the first loop through array1 (ie: if your values in array2 and array3 are maybe stored as "NN-name" and "NN-address" where NN is the id from the first array and you need to find the corresponding values and remove the NN- with a s// regex). Depends on how your data is structured though.
Another note is to check out Class::DBI and see if it might provide a nicer and more object oriented way of getting your data in.