perl DBI comma in query name - perl

I am using perl to talk to an sqlite database, and am getting an error. I think it is because some of the chemical names (stored in the variable $drugs) have commas in them (they also have many other 'strange characters'.
Any help would be appreciated, thanks.
**error message:**
- read in 57to find CIDs for
- Opened database successfully
- retrieving results for cyclo(L-Val-L-Pro)
- retrieving results for Sinapic_acid
- retrieving results for NHQ
- DBD::SQLite::db prepare failed: near ",": syntax error at get_drugName2IDinfo.sqlite.pl line 33.
- DBD::SQLite::db prepare failed: near ",": syntax error at get_drugName2IDinfo.sqlite.pl line 33.
line 33:
my $stmt = qq(SELECT * from chem_aliases WHERE alias LIKE '$drug%');
example drug names:
(2R,3R,4S,6R)-2-((5-hydroxy-2,2-dimethyl-3,4-dihydro-2H-benzo[h]chromen-6-yl)oxy)-6-(hydroxymethyl)tetrahydro-2H-pyran-3,4,5-triol
partial script:
my %HoDrugs;
while (my $line=<IN>){
chomp $line;
$HoDrugs{$line}=1;
}
close(IN);
print "read in\t".scalar(keys %HoDrugs)."to find CIDs for\n";
##
my $driver = "SQLite";
my $database = "/Users/alisonwaller/Documents/Typas/ext_data/STITCHv3.1/STITCHv3.1.sqlite.db";
my $dsn = "DBI:$driver:dbname=$database";
my $userid = "";
my $password = "";
my $dbh = DBI->connect($dsn, $userid, $password, { RaiseError => 1 })
or die $DBI::errstr;
print "Opened database successfully\n";
###
my $outfile="$in_drugNms.sq.plsCIDs.tab";
open (OUT,">",$outfile);
foreach my $drug (keys %HoDrugs){
my $stmt = qq(SELECT * from chem_aliases WHERE alias LIKE '$drug%');
my $sth = $dbh->prepare( $stmt );
my $rv = $sth->execute() or die $DBI::errstr;
if($rv < 0){
print $DBI::errstr;
}
while(my #row = $sth->fetchrow_array()) {
print "retrieving results for\t$drug\n";
print OUT join("\t",$drug,$row[0],$row[1],$row[2]) . "\n";
}
}
print "Operation done successfully\n";
$dbh->disconnect();

Have you tried using placeholders rather than just quoting the string yourself?
my $sth = $dbh->prepare( 'SELECT * from chem_aliases WHERE alias LIKE ?' );
my $rv = $sth->execute( $drug . '%' ) or die $DBI::errstr;

You could always try to use $drug =~ s/[[:punct:]]//g; before performing the query to try to remove punctuation characters?
If you don't want that, maybe replace them with spaces? $drug =~ s/,/ /g;

Related

How can I redirect the output of a SQL query to a file?

I need a little help in redirecting the output of a SQL query to a file. My code looks like this:
my $sth = $dbh->prepare(
"select count(parameter2),
parameter2 as file_type
from KCRT_TABLE_ENTRIES where request_id = $mycrnum
group by parameter2"
) or die "Can't prepare SQL statement: ", $dbh->errstr(), "\n";
$sth->execute > $mydir\\file_detail.txt
or die "Can't execute SQL statement: ", $sth->errstr(), "\n";
I've had to invent a lot of code as you don't show much of your program, but the program below gives you the rough idea
Once you've called execute you have to call one of the fetch methods to retrieve the data in whatever form is most useful to you. Here I've just asked for a reference to an array containing each row's data
Then it's simply a matter of opening the required file for output and printing the rows of data to it
I've removed the status checks on each DBI call and replaced it with the RaiseError flag which does the same thing automatically. I've also replaced the parameter $mycrnum in the SQL statement with a placeholder and passed its value to execute. That way DBI looks after any necessary quoting etc.
use strict;
use warnings;
use DBI;
my ($dsn, $user, $pass);
my ($mycrnum, $mydir);
my $dbh = DBI->connect($dsn, $user, $pass);
#{$dbh}{qw/ PrintError RaiseError /} = (0, 1);
my $sth = $dbh->prepare(
"SELECT COUNT(parameter2),
parameter2 AS file_type
FROM kcrt_table_entries
WHERE request_id = ?
GROUP BY parameter2"
);
$sth->execute($mycrnum);
open my $fh, '>', "$mydir/file_detail.txt" or die $!;
select $fh;
while ( my $row = $sth->fetchrow_arrayref ) {
printf "%5d %s\n", #$row;
}
After the execute, open the output file:
open my $of, ">", "$mydir\\file_detail.txt";
Then read each line (or row) in the results:
while ( #row = $sth->fetchrow_array ) {
Printing the output to the opened file handle:
print $of "#row\n"; # NO COMMA AFTER $of!
Close the while() loop:
}
Finally, close your opened file handle:
close $of;
Now your done.
Something like this, perhaps?
my $sth = $dbh->prepare(q{
select count(parameter2),
parameter2 as file_type
from KCRT_TABLE_ENTRIES where request_id = ?
group by parameter2
}) or die "Can't prepare SQL statement: ", $dbh->errstr(), "\n";
$sth->execute($mycrnum);
open my $OUT, '>', "$mydir/file_detail.txt" or die;
while (my #row = $sth->fetchrow_array) {
print $OUT #row, "\n"; # or whatever...
}
close $OUT;
$sth->finish;
This is a little bit of overkill, since you are only reading a single value, but it at least demonstrates a boilerplate for getting it done for future queries.
If you ever have a guaranteed single row, you can do something like this:
my ($val1, $val1) = $dbh->selectrow_array(q{
select foo, bar
});

How to print #, with 2 different type of string

I'm trying to write my first perl script. I find this a bit confusing since I only do C before this. I would like to print a string into a sqlfile and I'm having a problem to print a string that have #,. At the same time, I also need to print a variable.
This is my code:
.....
.....
Assume there is declaration and other function at the top
{
$dbh = DBI->connect("dbi:SQLite:dbname=data.db" or die "Connection error: $DBI::errstr\n";
my $stmt = qq(SELECT * from menu;);
my $sth = $dbh->prepare( $stmt ) ;
my $rv = $sth->execute() or die $DBI::errstr;
$m=0;
while(my #row = $sth->fetchrow_array()){
$ID[$m] = $row[0];
$NAME[$m] = $row[1];
$PASSWORd_FLG[$m] = $row[2];
$m++;
}
$k = $m;
my $stmt2 = qq(SELECT * from menuitem;);
my $sth2 = $dbh->prepare( $stmt2 );
my $rv = sth2->execute() or die $DBI::errstr;
$b=0;
while(my #row = $sth2->fetchrow_arrray()){
$ITEM_ID[$b] = $row[0];
$ITEM_MENU[$b] = $row[1];
$ITEM_NAME[$b] = $row[2];
$b++;
}
write_menu_to_sql_file($k,$c);
$dbh->disconnect()'
}
sub write_menu_to_sql_file{
my $k = #_;
my $c = #_;
print (FILE, ">>$sqlfile") or die $!;
print FILE ('declare #last_menu_id int');
for($m=0;m<$k;$m++)
{
print FILE ("insert into My_Menu (NAME, PASSWORD) values ('$NAME[$m]', '$PASSWORD_FLG[$m]')\n");
print FILE('set #last_menu_id = (select ##IDENTITY)');
for($b=0;$b<$c;$b++){
print FILE("insert into My_Item_Menu (NAME,ID,ITEM_MENU) values ('$ITEM_NAME[$b]', #last_menu_id, $ITEM_MENU[$b])\n");
}
}
}
The output in the sqlfile will be:
declare #last_menu_id int
insert into My_Menu (NAME, PASSWORD) values ('Fikrie', 'Y')
insert into My_ITEM_Menu (NAME. ID, ITEM_MENU) values ('WORK', ,'Fikrie')
.......
.......
It will loop for all the data
As you can see, the #last_menu_id is declared as a variable, which I do not want. Instead I just want it to be as a string. So I tried changing all the word to use ' ' quote. But then, It doesnt recognize the other variable. How do I print a variable and a string that have # in 1 line? I tried to write the code like this, but it gives error.
print FILE("insert into My_Item_Menu (NAME,ID,ITEM_MENU) values ('$ITEM_NAME[$b]', "'#last_menu_id,'" $ITEM_MENU[$b])\n");
This is the error that I got:
String found where operator expected near ""insert into My_Item_Menu (NAME,ID,ITEM_MENU) values ('$ITEM_NAME[$b]', "'#last_menu_id,'"
(Missing operator before '#last_menu_id,'?)
Possible unintended interpolation of #last_menu_id in string at ./testdb.pl line.....
You can escape '#' character and try. Something like this: (Note the backslash before the # symbol)
print FILE("insert into My_Item_Menu (NAME,ID,ITEM_MENU) values ('$ITEM_NAME[$b]', "'\#last_menu_id,'" $ITEM_MENU[$b])\n");

CSV import to MySQL

Hi I keep getting an error when trying to run the following perl script to import a csv file into an existing mysql database table. Every time I run it I get the message "Died at /home/perl/dep_import_2.pl line 10.
Any help would be appreciated
Thanks
#!/usr/bin/perl
use DBI;
use DBD::mysql;
use warnings "all";
if ($#ARGV != 0) {
print "Usage: dep_import_2.pl filename\n";
die;
}
$filename = $ARGV[0];
# MySQL CONFIG VARIABLES
$host = "localhost";
$user = "standard";
$pw = "standard";
$database = "data_track";
$dsn = "DBI:mysql:database=" . $database . ";host=" . $host;
$dbh = DBI->connect($dsn, $user, $pw)
or die "Can't connect to the DB: $DBI::errstr\n";
print "Connected to DB!\n";
open FILE, "/home/dep/new_study_forms_2.csv", $filename or die $!;
$_ = <FILE>;
$_ = <FILE>;
while (<FILE>) {
#f = split(/,/, $_);
$sql = "INSERT INTO dep (date, subject, weight, size, time, hi_pre, hi_post, hi_afternoon, hi_test, actical_on, actical_off, saggital_1, saggital_2, crown_heel1, crown_heel2, crown_rump1, crown_rump2, scan, record_number, tap, sample, dye, left_chip, right_chip) VALUES('$f[0]', '$f[1]', '$f[2]', '$f[3]' '$f[4]', '$f[5]', '$f[6]', '$f[7]', '$f[8]', '$f[9]', '$f[10]', '$f[11]', '$f[12]', '$f[13]', '$f[14]', '$f[15]', '$f[16]', '$f[17]', '$f[18]', '$f[19]', '$f[20]', '$f[21]', '$f[22]', '$f[23]')";
print "$sql\n";
my $query = $dbh->do($sql);
}
There are a few issues with your code. First, and most importantly, you are not using
use strict;
use warnings;
This is bad because you will not get information about errors in your code without them.
As others have pointed out, the reason the script dies is because $#ARGV is not zero. Meaning that you have either passed too few or too many arguments to the script. The arguments to the script must be exactly one, like the usage statement says.
However, that would not solve your problem, because your open statement below is screwed up. My guess is that you tried to add your file name directly. This line:
open FILE, "/home/dep/new_study_forms_2.csv", $filename or die $!;
It will probably give you the error unknown open() mode .... It should probably be
open FILE, "<", $filename or die $!;
And then you pass /home/dep/new_study_forms_2.csv to the script on the command line, assuming that is the correct file to use.
Also, in your query string, you should not interpolate variables, you should use placeholders, as is described in the documentation for DBI. The placeholders will take care of the quoting for you and avoid any data corruption. To make your query line a bit simpler, you can do something like:
my $sth = $dbh->prepare(
"INSERT INTO dep (date, subject, weight, size, time, hi_pre, hi_post,
hi_afternoon, hi_test, actical_on, actical_off, saggital_1, saggital_2,
crown_heel1, crown_heel2, crown_rump1, crown_rump2, scan, record_number,
tap, sample, dye, left_chip, right_chip)
VALUES(" . join(",", ("?") x #f) . ")");
$sth->execute(#f);
Here's a script which uses Text::CSV to properly parse CSV. It assumes that the first row contains column names, and then loads the CSV in batches, commiting after every 100 inserts. Every parameter (user, password, database) is configurable via command-line options. Usage is an in-line POD document.
#!/usr/bin/env perl
use strict;
use warnings qw(all);
use DBI;
use Getopt::Long;
use Pod::Usage;
use Text::CSV_XS;
=pod
=head1 SYNOPSIS
dep_import_2.pl --filename=file.csv --host=localhost --user=standard --pw=standard --database=data_track
=head1 DESCRIPTION
Loads a CSV file into the specified MySQL database.
=cut
my $host = 'localhost';
my $user = 'standard';
my $pw = 'standard';
my $database = 'data_track';
my $commit = 100;
GetOptions(
'help' => \my $help,
'filename=s' => \my $filename,
'host=s' => \$host,
'user=s' => \$user,
'pw=s' => \$pw,
'database=s' => \$database,
'commit=i' => \$commit,
) or pod2usage(q(-verbose) => 1);
pod2usage(q(-verbose) => 2) if $help;
my $dbh = DBI->connect("DBI:mysql:database=$database;host=$host", $user => $pw)
or die "Can't connect to the DB: $DBI::errstr";
my $csv = Text::CSV_XS->new
or die "Text::CSV error: " . Text::CSV->error_diag;
open(my $fh, '<:utf8', $filename)
or die "Can't open $filename: $!";
my #cols = #{$csv->getline($fh)};
$csv->column_names(\#cols);
my $query = "INSERT INTO dep (#{[ join ',', #cols ]}) VALUES (#{[ join ',', ('?') x (scalar #cols) ]})";
my $sth = $dbh->prepare($query);
my $i = 0;
while (my $row = $csv->getline_hr($fh)) {
$sth->execute(#{$row}{#cols});
$dbh->commit if ((++$i % $commit) == 0);
}
$dbh->commit;
$dbh->disconnect;
$csv->eof or $csv->error_diag;
close $fh;

how to pass in sql query input file to a perl dbi sub routine

I like to create a generic perl script that will input sql query from a separate file and use it with perl dbi (subroutine), rather than hardcoding the statement. Can someone show me an example how to do this?
For example I have this in a sub routine:
sub get_val
{
my $sth = $dbh->prepare(q{SELECT count(*) AS COUNT FROM TEST1) ||
die ("Can't connect: ".$DBI::errstr."\n");
$sth->execute;
my $row = $sth->fetchrow_hashref;
$sth->finish;
return $row->{COUNT};
}
This would be the general idea:
$/ = ';';
open FH, "< file.sql";
while (<FH>) {
$dbh->do($_);
# or:
# my $sth = $dbh->prepare($_);
# $sth->execute();
}
close FH;
Of course this won't necessarily handle comments, or ; characters in quoted strings, etc. But this should head you in the right direction.
Or if you know the file will only contain a single statement:
undef $/;
open FH, "< file.sql";
my $sth = $dbh->prepare(<FH>);
close FH;
$sth->execute();

Why does DBD::CSV complain about "Loose unescaped quote"?

Why does reading from __DATA__ work and reading from the file doesn't (Loose unescaped quote)?
#!/usr/bin/env perl
use warnings; use strict; use 5.010;
use DBI;
my $table = 'klassik_CD.csv';
print qx(cat $table);
print qq{\n"data" or "Enter" : };
chomp( my $aw = <> );
if ( $aw eq 'data' ) {
$table = 'te_mp_fi_le.csv';
open my $fh, '>', $table or die $!;
while ( defined( my $row = <DATA> ) ) {
print $fh $row;
}
close $fh or die $!;
}
my $dbh = DBI->connect( "dbi:CSV:", { RaiseError => 1 } );
$dbh->{csv_tables}{$table} = { col_names => [], sep_char => ';' };
my $sth = $dbh->prepare( "SELECT * FROM $table" );
$sth->execute;
while ( defined( my $row = $sth->fetchrow_hashref ) ) {
say $row->{col1};
}
__DATA__
1;"Kammermusik fuer Blaeser";16;"DG";"eloquence";"dc129610"
2;"Requiem – Laudate Dominum Exultate, jubilate";19;"DG";"eloquence";"0a11f513"
Output: "data"
1;"Kammermusik fuer Blaeser";16;"DG";"eloquence";"dc129610"
2;"Requiem – Laudate Dominum Exultate, jubilate";19;"DG";"eloquence";"0a11f513"
"data" or "Enter" : data
Kammermusik fuer Blaeser
Requiem – Laudate Dominum Exultate, jubilate
Output: "Enter"
1;"Kammermusik fuer Blaeser";16;"DG";"eloquence";"dc129610"
2;"Requiem – Laudate Dominum Exultate, jubilate";19;"DG";"eloquence";"0a11f513"
"data" or "Enter" :
DBD::CSV::st execute failed:
Execution ERROR: Error 2034 while reading file ./klassik_CD.csv: EIF - Loose unescaped quote at /usr/local/lib/perl5/site_perl/5.10.1/DBD/CSV.pm line 220
.
[for Statement "SELECT * FROM klassik_CD.csv"] at ./zzzzzzzzzz.pl line 27.
DBD::CSV::st fetchrow_hashref failed: Attempt to fetch row without a preceeding execute () call or from a non-SELECT statement [for Statement "SELECT * FROM klassik_CD.csv"] at ./zzzzzzzzzz.pl line 28.
When I rename the file from "klassik_CD.csv" to "klassik_cd.csv" (all lowercase) it works (though there was no such message as "file not found" ).