Is there any easier way to add char to the beginning and end of array elements in perl.
Ex:my $str ='table1,table2,table3'
my #arry = split (/,/,$str)
I like the output should look like
'table1','table2','table3'
So it can be used in sql query
Thanks.
join(',', map "'$_'", #arry)
is what you are asking for.
But it's much better to use placeholders:
my $str = 'table1,table2,table3';
my #arry = split(/,/, $str);
if (#arry) {
my $query = 'select * from tablename where colname in (' . join(',',('?') x #arry) . ')';
my $sth = $dbh->prepare($query);
$sth->execute(#arry);
...
}
If you are going to use DBI (https://metacpan.org/module/DBI) to work with your database, it can do it for you if you use placeholders in your queries, so that you don't have to quote values.
For example, if you want to insert something into a table:
$dbh->do("INSERT INTO table VALUES(?, ?, ?)", undef, #arry)
or die $dbh->errstr;
where #arry is an array containing exactly 3 values to be inserted into a table.
More on this can be found here https://metacpan.org/module/DBI#Placeholders-and-Bind-Values
Related
# I am saving output to an array and the array looks like this:-
60=20130624-09:45:02.046|21=1|38=565|52=20130624-09:45:02.046|35=D|10=085|40=1|9=205|100=MBTX|49=11342|553=2453|34=388|1=30532|43=Y|55=4323|54=1|56=MBT|11=584|59=0|114=Y|8=FIX.4.4|
# Then i converted this array to scalar variable like this:-
$scal=join('' , #arr);
# And now I am trying to save this into db:-
my $st = qq(INSERT INTO demo (fix)
VALUES ($scal));
my $r = $dbh->do($st) or die $DBI::errstr;
#And my table schema is:-
CREATE TABLE demo (fix varchar);
And I keep getting errors :- DBD::SQLite::db do failed: near ":45": syntax error at pdb.pl line 92, <STDIN> line 1.
DBD::SQLite::db do failed: near ":45": syntax error at pdb.pl line 92, <STDIN> line 1.
Any help will be appreicated
The way you denote your array is a bit weird. Usually you would write it as
my #arr = ( '60=20130624-09:45:02.046',
'21=1',
'38=565',
... );
or whatever your actual content is. But this is not the problem here because you flatten it to the string $scal anyway.
One way to insert this string into your DB is to put ticks (') around it:
my $st = qq(INSERT INTO demo (fix) VALUES ('$scal'));
my $r = $dbh->do($st) or die $DBI::errstr;
But this is bad because it's vulnerable to SQL injection (http://imgs.xkcd.com/comics/exploits_of_a_mom.png).
Consider the case your string is foo'); delete from demo; --. The final result would then be
INSERT INTO demo (fix) VALUES ('foo'); delete from demo; --')
The second reason why this is bad: Your string could contain ticks ($scal="foo's bar") and that also would mess up the resulting INSERT statement:
INSERT INTO demo (fix) VALUES ('foo's bar');
Conclusion: it's always better to use parameterized queries:
my $st = 'INSERT INTO demo (fix) VALUES (?)';
my $r = $dbh->do($st, undef, $scal) or die $DBI::errstr;
The undef is for additional SQL options (I've rarely seen anything different from undef here). The following parameters are replaced for the ?s in the statement. The DB driver does all the quoting for you. The more ? you use, the more parameters you must supply to do():
my $st = 'INSERT INTO sample_tbl (col1, col2, col3) VALUES (?, ?, ?)';
my $r = $dbh->do($st, undef, 'foo', 42, $scal) or die $DBI::errstr;
I know I can do this with interpolation. Can I do it using placeholders?
I am getting this error:
DBD::Pg::st execute failed: ERROR: invalid input syntax for integer: "{"22,23"}" at ./testPlaceHolders-SO.pl line 20.
For this script:
#!/usr/bin/perl -w
use strict;
use DBI;
# Connect to database.
my $dbh = DBI->connect("dbi:Pg:dbname=somedb;host=localhost;port=5432", "somedb", "somedb");
my $typeStr = "22,23";
my #sqlParms = [ $typeStr ];
my $sqlStr = << "__SQL_END";
SELECT id
FROM states
WHERE typeId in (?)
ORDER BY id;
__SQL_END
my $query = $dbh->prepare($sqlStr);
$query->execute(#sqlParms);
my $id;
$query->bind_columns(\$id);
# Process rows
while ($query->fetch())
{
print "Id: $id\n";
}
Is there a way around it besides interpolation?
DBD::PG has support for PostgreSQL arrays, so you can simply write a query like this:
WHERE typeid = ANY( ARRAY[1,2,3] )
or, with a parameter...
WHERE typeid = ANY(?)
Then just use the array support
my #targets = (1,2,3);
# ...
$query->execute(\#targets);
Posting comment as answer, as requested.
Generate your own placeholder string. Like so:
my #nums = (22,23);
my $placeholder = join ",", ("?") x #nums;
$query->execute(#nums);
Yes. You must use placeholders for each value, such as IN (?, ?, ?). You can however generate the correct number of question marks using something like this (untested):
my #values = (22, 23, ...);
# will produce "?, ?, ..."
my $in = join ", ", ("?") x #values;
my $sqlStr = "SELECT id FROM states WHERE typeId in ($in) ORDER BY id;";
my $query = $dbh->prepare($sqlStr);
$query->execute(#values);
Note that if you use an ORM such as DBIx::Class instead, this sort of ugly hack gets abstracted away.
You have to build the SQL statement with the correct number of question marks and then set the parameter values. There is no way to bind a list to a single question mark.
I have this query select * from table where ID in (1,2,3,5...)
How is it possible to build this query with the DBI using placeholders ?
for example :
my #list = (1, 2, 3, 4, 5);
my $sql = "select * from table where ID in (?)";
$sth->prepare($sql);
$sth->execute();
What argument should I send to execute? Is it a list or a string separated by , or something else?
This should build your query dynamically according to the number of items in your array
my #list =(1,2,3,4,5);
my $sql ="select * from table where ID in (#{[join',', ('?') x #list]})";
It's not possible in that way. You need to specify a placeholder for each item in your array:
my #list = (1,2,3,4,5);
my $sql = "select * from table where ID in (?,?,?,?,?)";
$sth->prepare($sql);
$sth->execute(#list);
If your #list is not a fixed size, you need to build the $sql with the proper number of placeholders.
Quoting DBI documentation:
Also, placeholders can only represent single scalar values. For example, the following statement won't work as expected
for more than one value:
SELECT name, age FROM people WHERE name IN (?) # wrong
SELECT name, age FROM people WHERE name IN (?,?) # two names
Rewrite to:
my $sql = 'select * from table where ID in ( ?, ?, ?, ?, ? )';
$sth->prepare($sql);
$sth->execute(#list);
If you are using DBI to access a PostgreSQL database with the DBD::Pg driver, you can use:
my #list = (1, 2, 3, 4, 5);
my $sql = "select * from table where ID = ANY(?::INT[]);";
$sth->prepare ($sql);
$sth->execute (\#list);
Unless you know the exact number of elements you cannot use placeholders. Try this:
my #list = (1, 2, 3, 4, 5); # any number of elements
my $in = join(',', map { $dbh->quote($_) } #list);
my $sql = "select * from table where someid IN ($in)";
If you switch to DBIx::Simple you can just say:
$db->query('INSERT INTO foo VALUES (??)', $foo, $bar, $baz);
?? Means "as many as needed"
Edit:
Actually, I was a little too optimistic: "If the string (??) is present in the query, it is replaced with a list of as many question marks as #values."
So this does not seem to work:
$db->query( "SELECT * FROM foo WHERE id IN (??) AND stuff=?", #ids, $stuff )
Still useful though..
For the curious, the code in the module is:
# Replace (??) with (?, ?, ?, ...)
sub _replace_omniholder {
my ($self, $query, $binds) = #_;
return if $$query !~ /\(\?\?\)/;
my $omniholders = 0;
my $q = $self->{dbd} =~ /mysql/ ? $quoted_mysql : $quoted;
$$query =~ s[($q|\(\?\?\))] {
$1 eq '(??)'
? do {
Carp::croak('There can be only one omniholder')
if $omniholders++;
'(' . join(', ', ('?') x #$binds) . ')'
}
: $1
}eg;
}
I found a sure way for this to work summarizing all of the above advice. My Production query (I posted a much simpler version here) uses IN <>, where neither the codes nor their quantity is unknown. It could be a single Code (e.g. FIN), or a series of them (FITLC FITLD FITLU FITSC FITSD FITSU MDYLC MDYLD MDYLU). Some function returns that as a list.
The code that makes this happen is
#codes = get_muni_evcode( $category );
my $in = join( ', ', ('?') x #codes );
print "\n\nProcessing Category: $category --> Codes: #codes .. in: $in\n";
my $sql = "select distinct cusip9
from material_event
where event_date between (trunc(sysdate) - 1) + 2/3 and trunc(sysdate) + 2/3
and event_code in ($in)";
my $sth2 = $dbh->prepare($sql);
$sth2->execute( #codes );
while (my $s2 = $sth2->fetchrow_hashref('NAME_lc'))
{
my $cusip9 = $s2->{cusip9};
print "$cusip9\t";
.................. further processing ..............
}
The result sample:
Processing Category: RatingChange --> Codes: FITLC FITLD FITLU FITSC FITSD FITSU MDYLC MDYLD MDYLU MDYSC MDYSD MDYSU SPLD SPLPR SPLU SPSD SPSPR SPSU .. in: ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?
359496HQ2 359496GB6 359496GH3 359496GL4 359496HU3 359496HS8 359496HA7 359496HF6 359496GM2 359496HM1 359496HR0 359496HT6 359496GY6 359496GJ9 359496HL3 359496GU4 359496HK5 359496HN9 359496HP4 359496GW0 359496GZ3 359496HC3 359496GC4 359496GK6 359496GP5 359496GV2 359496GX8 359496GN0
I'm extremely grateful to everybody who posted their ideas here that finally made me find the right way to do this. It must be a pretty common problem I think.
I need to insert values in database using Perl's DBI module. I have parsed a file to obtain these values and hence these values are present in an arrays, say #array1, #array2, #array3. I know how to insert one value at a time but not from an arrays.
I know insert one value at a time:
$dbh = DBI->connect("dbi:Sybase:server=$Srv;database=$Db", "$user", "$passwd") or die "could not connect to database";
$query= "INSERT INTO table1 (id, name, address) VALUES (DEFAULT, tom, Park_Road)";
$sth = $dbh->prepare($query) or die "could not prepare statement\n";
$sth-> execute or die "could not execute statement\n $command\n";
I am not sure if I have array1 containing ids, array2 containing names, and array3 containing address, how would I insert values.
Since you have parallel arrays, you could take advantange of execute_array:
my $sth = $dbh->prepare('INSERT INTO table1 (id, name, address) VALUES (?, ?, ?)');
my $num_tuples_executed = $sth->execute_array(
{ ArrayTupleStatus => \my #tuple_status },
\#ids,
\#names,
\#addresses,
);
Please note that this is a truncated (and slightly modified) example from the documentation. You'll definitely want to check out the rest of it if you decide to use this function.
Use placeholders.
Update: I just realized you have parallel arrays. That is really not a good way of working with data items that go together. With that caveat, you can use List::MoreUtils::each_array:
#!/usr/bin/perl
use strict; use warnings;
use DBI;
use List::MoreUtils qw( each_array );
my $dbh = DBI->connect(
"dbi:Sybase:server=$Srv;database=$Db",
$user, $passwd,
) or die sprintf 'Could not connect to database: %s', DBI->errstr;
my $sth = $dbh->prepare(
'INSERT INTO table1 (id, name, address) VALUES (?, ?, ?)'
) or die sprintf 'Could not prepare statement: %s', $dbh->errstr;
my #ids = qw( a b c);
my #names = qw( d e f );
my #addresses = qw( g h i);
my $it = each_array(#ids, #names, #address);
while ( my #data = $it->() ) {
$sth->execute( #data )
or die sprintf 'Could not execute statement: %s', $sth->errstr;
}
$dbh->commit
or die sprintf 'Could not commit updates: %s', $dbh->errstr;
$dbh->disconnect;
Note that the code is not tested.
You might also want to read the FAQ: What's wrong with always quoting "$vars"?.
Further, given that the only way you are handling error is by dying, you might want to consider specifying { RaiseError => 1 } in the connect call.
How could you not be sure what your arrays contain? Anyway the approach would be the iterate through one array and assuming the other arrays have corresponding values put those into the insert statement
Another way would be to use a hash as an intermediate storage area. IE:
my $hash = {};
foreach(#array1) {
$hash->{id} = $array1[$_];
$hash->{name} = $array2[$_];
$hash->{address} = $array3[$_];
}
foreach( keys %$hash ) {
$sql = "insert into table values(?,?,?)";
$sth = $dbh->prepare($sql) or die;
$sth->execute($hash->{id}, $hash->{name}, $hash->{address}) or die;
}
Though again this depends on the three arrays being synced up. However you could modify this to do value modifications or checks or greps in the other arrays within the first loop through array1 (ie: if your values in array2 and array3 are maybe stored as "NN-name" and "NN-address" where NN is the id from the first array and you need to find the corresponding values and remove the NN- with a s// regex). Depends on how your data is structured though.
Another note is to check out Class::DBI and see if it might provide a nicer and more object oriented way of getting your data in.
I'm writing a subroutine for DBI updates, and are having some trouble figuring out how to add placeholder and stuff...
I have this:
sub row_update {
my $table = shift;
my %updates = #_;
my $placeholders = ...
$dbh->do("UPDATE $table SET (foo) WHERE (bar)") etc...
}
Any ideas?
I just want a simple update function where I can send in x number of arguments (as a hash).
Thanks in advance!
Something like this might be Good Enough:
sub update {
my ($dbh, $args) = #_;
my $table = $args->{table} || die 'need table';
my $updates = $args->{updates} || die 'need updates';
my #cols = keys %$updates;
my $query = 'UPDATE $table SET '.
(join ', ', map { '$_ = ?' } #cols)
($args->{where} ? ' WHERE '. $args->{where} : '');
my $sth = $dbh->prepare($query);
$sth->execute(map { $updates->{$_} } #cols);
return $sth;
}
Use it like:
my $sth = update $dbh, {
table => 'foo',
updates => {
col1 => 'new_value',
col2 => 'another_value',
},
where => 'id=42',
};
Really, though, you want to look into using an ORM like
DBIx::Class. It will do a much better job of building queries than
string manipulation like this will.
(Rewriting the where clause to be parameterized is left as an exercise to the reader. You also need to quote the update keys and table name. See why people use ORMs?)
Edit: Thinking about this a bit more, you might like DBIx::Simple combined with SQL::Abstract. This will take less configuration effort than an ORM, but still give you many of the benefits.
If I understand the question correctly, it sounds like you're after SQL::Abstract. First, we create an SQL::Abstract object:
use SQL::Abstract;
my $sql = SQL::Abstract->new;
Now, as an example, we'll use it to insert some data into a table:
my %record = (
FirstName => 'Buffy',
LastName => 'Summers',
Address => '1630 Revello Drive',
City => 'Sunnydale',
State => 'California',
Occupation => 'Student',
Health => 'Alive',
);
my ($stmt, #bind) = $sql->insert(’staff’,\%record);
This results in:
$stmt = "INSERT INTO staff
(FirstName, LastName, Address, City,
State, Occupation, Health)
VALUES (?, ?, ?, ?, ?, ?, ?)";
#bind = ('Buffy','Summers','1630 Revello Drive',
'Sunnydale',’California','Student','Alive');
The nice thing about this is we can pass it directly to DBI:
$dbh->do($stmt, undef, #bind);
Of course, you want to be updating records, not just inserting them. Luckily, this is also quite easy:
my $table = 'People';
my %new_fields = (
Occupation => 'Slayer',
Health => 'Dead',
);
my %where = (
FirstName => 'Buffy',
LastName => 'Summers',
);
my ($stmt, #bind) = $sql->update($table, \%new_fields, \%where);
$dbh->do($stmt, undef, #bind);
This produces:
$stmt = 'UPDATE People SET Health = ?, Occupation = ?
WHERE ( FirstName = ? AND LastName = ? )';
#bind = ('Dead', 'Slayer', 'Buffy', 'Summers');
If you're after more information about SQL::Abstract, I recommend you look at its CPAN page. There's also a chapter in Perl Training Australia's Database Programming with Perl manual, which are freely available from our course notes page.
All the best,
Paul
Disclaimer: I'm managing director of Perl Training Australia, and therefore think that our course notes are pretty good.
Others suggested the usual "build a query with the right number of '?'s" approach.
For most queries like this the DBI->quote method is forgotten, but it can keep the code simpler and in most cases it's not any slower than the "proper" placeholder approach.
1) Use DBI->quote instead of place holders to build the query. For example for a simple select:
my $sql = "select foo from bar where baz in ("
. join(",", map { DBI->quote($_) } #bazs)
. ")";
my $data = $dbh->selectall_arrayref($sql);
2) As jrockway suggested - use an ORM to do this sort of low level stuff for you. DBIx::Class or Rose::DB::Object, for example.
The NUM_OF_PARAMS attribute may be of assistance:
"NUM_OF_PARAMS" (integer, read-only)
The number of parameters (placeholders) in the prepared
statement.
So for instance, I've got a script to run arbitrary SQL from the command line that uses this code:
my $s = $h->prepare($_);
for my $i (1..$s->{NUM_OF_PARAMS}){
my $param = shift #ARGV;
$s->bind_param($i, $param);
print LOG "Bind param $i using $param.\n"
or die "can't append to $opt{log}: $!";
}
$s->execute();
I can't say I've used the other modules that have been suggested, so they may do a better job.