I need to manipulate some records in a DB and write their values to another table. Some of these values have an '&' in the string, i.e. 'Me & You'. Short of finding all of these values and placing a \ before any &'s, how can insert these values into a table w/o oracle choking on the &?
Use placeholders. Instead of putting '$who' in your SQL statement, prepare with a ? there instead, and then either bind $who, or execute with $who as the appropriate argument.
my $sth = $dbh->prepare_cached('INSERT INTO FOO (WHO) VALUES (?)');
$sth->bind_param(1, $who);
my $rc = $sth->execute();
This is safer and faster than trying to do it yourself. (There is a "quote" method in DBI, but this is better than that.)
This is definitely a wheel you don't need to reinvent. If you are using DBI, don't escape the input; use placeholders.
Example:
my $string = "database 'text' with &special& %characters%";
my $sth = $dbh->prepare("UPDATE some_table SET some_column=?
WHERE some_other_column=42");
$sth->execute($string);
The DBD::Oracle module (and all the other DBD::xxxxx modules) have undergone extensive testing and real world use. Let it worry about how to get your text inserted into the database.
Related
I have a table test(id,name).
I need to insert values like: user's log, 'my user', customer's.
insert into test values (1,'user's log');
insert into test values (2,''my users'');
insert into test values (3,'customer's');
I am getting an error if I run any of the above statements.
If there is any method to do this correctly please share. I don't want any prepared statements.
Is it possible using sql escaping mechanism?
String literals
Escaping single quotes ' by doubling them up → '' is the standard way and works of course:
'user's log' -- incorrect syntax (unbalanced quote)
'user''s log'
Plain single quotes (ASCII / UTF-8 code 39), mind you, not backticks `, which have no special purpose in Postgres (unlike certain other RDBMS) and not double-quotes ", used for identifiers.
In old versions or if you still run with standard_conforming_strings = off or, generally, if you prepend your string with E to declare Posix escape string syntax, you can also escape with the backslash \:
E'user\'s log'
Backslash itself is escaped with another backslash. But that's generally not preferable.
If you have to deal with many single quotes or multiple layers of escaping, you can avoid quoting hell in PostgreSQL with dollar-quoted strings:
'escape '' with '''''
$$escape ' with ''$$
To further avoid confusion among dollar-quotes, add a unique token to each pair:
$token$escape ' with ''$token$
Which can be nested any number of levels:
$token2$Inner string: $token1$escape ' with ''$token1$ is nested$token2$
Pay attention if the $ character should have special meaning in your client software. You may have to escape it in addition. This is not the case with standard PostgreSQL clients like psql or pgAdmin.
That is all very useful for writing PL/pgSQL functions or ad-hoc SQL commands. It cannot alleviate the need to use prepared statements or some other method to safeguard against SQL injection in your application when user input is possible, though. #Craig's answer has more on that. More details:
SQL injection in Postgres functions vs prepared queries
Values inside Postgres
When dealing with values inside the database, there are a couple of useful functions to quote strings properly:
quote_literal() or quote_nullable() - the latter outputs the unquoted string NULL for null input.
There is also quote_ident() to double-quote strings where needed to get valid SQL identifiers.
format() with the format specifier %L is equivalent to quote_nullable().
Like: format('%L', string_var)
concat() or concat_ws() are typically no good for this purpose as those do not escape nested single quotes and backslashes.
According to PostgreSQL documentation (4.1.2.1. String Constants):
To include a single-quote character within a string constant, write
two adjacent single quotes, e.g. 'Dianne''s horse'.
See also the standard_conforming_strings parameter, which controls whether escaping with backslashes works.
This is so many worlds of bad, because your question implies that you probably have gaping SQL injection holes in your application.
You should be using parameterized statements. For Java, use PreparedStatement with placeholders. You say you don't want to use parameterised statements, but you don't explain why, and frankly it has to be a very good reason not to use them because they're the simplest, safest way to fix the problem you are trying to solve.
See Preventing SQL Injection in Java. Don't be Bobby's next victim.
There is no public function in PgJDBC for string quoting and escaping. That's partly because it might make it seem like a good idea.
There are built-in quoting functions quote_literal and quote_ident in PostgreSQL, but they are for PL/PgSQL functions that use EXECUTE. These days quote_literal is mostly obsoleted by EXECUTE ... USING, which is the parameterised version, because it's safer and easier. You cannot use them for the purpose you explain here, because they're server-side functions.
Imagine what happens if you get the value ');DROP SCHEMA public;-- from a malicious user. You'd produce:
insert into test values (1,'');DROP SCHEMA public;--');
which breaks down to two statements and a comment that gets ignored:
insert into test values (1,'');
DROP SCHEMA public;
--');
Whoops, there goes your database.
In postgresql if you want to insert values with ' in it then for this you have to give extra '
insert into test values (1,'user''s log');
insert into test values (2,'''my users''');
insert into test values (3,'customer''s');
you can use the postrgesql chr(int) function:
insert into test values (2,'|| chr(39)||'my users'||chr(39)||');
When I used Python to insert values into PostgreSQL, I also met the question: column "xxx" does not exist.
The I find the reason in wiki.postgresql:
PostgreSQL uses only single quotes for this (i.e. WHERE name = 'John'). Double quotes are used to quote system identifiers; field names, table names, etc. (i.e. WHERE "last name" = 'Smith').
MySQL uses ` (accent mark or backtick) to quote system identifiers, which is decidedly non-standard.
It means PostgreSQL can use only single quote for field names, table names, etc. So you can not use single quote in value.
My situation is: I want to insert values "the difference of it’s adj for sb and it's adj of sb" into PostgreSQL.
How I figure out this problem:
I replace ' with ’, and I replace " with '. Because PostgreSQL value does not support double quote.
So I think you can use following codes to insert values:
insert into test values (1,'user’s log');
insert into test values (2,'my users');
insert into test values (3,'customer’s');
If you need to get the work done inside Pg:
to_json(value)
https://www.postgresql.org/docs/9.3/static/functions-json.html#FUNCTIONS-JSON-TABLE
You must have to add an extra single quotes -> ' and make doubling quote them up like below examples -> ' ' is the standard way and works of course:
Wrong way: 'user's log'
Right way: 'user''s log'
problem:
insert into test values (1,'user's log');
insert into test values (2,''my users'');
insert into test values (3,'customer's');
Solutions:
insert into test values (1,'user''s log');
insert into test values (2,'''my users''');
insert into test values (3,'customer''s');
I have a SQL query
select name from Employee
Output :
Sharma's
How can I store this output in perl string.
I tried below :
$sql =qq {select Name from Employee};
$Arr = &DataBaseQuery( $dbHandle, $sql );
$name = $Arr;
But when I print $name I get output as
Sharma::s
How can I store the single quote in the $name.
First of all, non of standard DBI/DBD exibits behavior you listed, in my experience.
Without knowing details of what DataBaseQuery() does it's impossible to answer conclusively, but a plausible theory can be formed:
Apostrophe is a valid package separator in Perl, equivalent to "::".
Reference: perldoc perlmod
The old package delimiter was a single quote, but double colon is now the preferred delimiter, in part because it's more readable to humans, and in part because it's more readable to emacs macros. It also makes C++ programmers feel like they know what's going on--as opposed to using the single quote as separator, which was there to make Ada programmers feel like they knew what was going on. Because the old-fashioned syntax is still supported for backwards compatibility, if you try to use a string like "This is $owner's house" , you'll be accessing $owner::s ; that is, the $s variable in package owner , which is probably not what you meant. Use braces to disambiguate, as in "This is ${owner}'s house" .
perl -e 'package A::B; $A::B=1; 1;
package main;
print "double-colon: $A::B\n";
print "apostrophe: $A'"'"'B\n";'
double-colon: 1
apostrophe: 1
I have a strong suspicion something within your own libraries inside DataBaseQuery() call was written to be "smart" and to convert apostrophes to double-colons because of this.
If you can't figure out root cause, you can always do one of the following:
Write your own DB wrapper
Assuming your text isn't likely to contain "::", run a regex to fix s#::#'#g; on all results from DataBaseQuery() (likely, in a function serving as a wrapper-replacement for DataBaseQuery())
$sth = $dbh->prepare($sql);
$sth->execute();
$sth->{NAME};
But how do you do that when:
$hr = $dbh->selectall_hashref($sql,'pk_id');
There's no $sth, so how do you get the $sth->{NAME}? $dbh->{NAME} doesn't exist.
When you're looking at a row, you can always use keys %$row to find out what columns it contains. They'll be exactly the same thing as NAME (unless you change FetchHashKeyName to NAME_lc or NAME_uc).
You can always prepare and execute the handle yourself, get the column names from it, and then pass the handle instead of the sql to selectall_hashref (e.g. if you want the column names but the statement may return no rows). Though you may as well call fetchall_hashref on the statement handle.
This is a standard inserting example from DBI manual:
my $query = q{
INSERT INTO sales (product_code, qty, price) VALUES (?, ?, ?)
};
my $sth = $dbh->prepare($query) or die $dbh->errstr;
while (<>) {
chomp;
my ($product_code, $qty, $price) = split /,/;
$sth->execute($product_code, $qty, $price) or die ($query . " " . $dbh->errstr);
}
$dbh->commit or die $dbh->errstr;
I modified it a bit, so I can see on die which query failed (die ($query . " " . $dbh->errstr)). Still I'd liked to see the query with bound values (as it was executed). How to get it?
Edit
Btw, i found a awkward way to see query with bound values too: you have to make syntax error in query. For example, if i change query above like that:
my $query = q{
xINSERT INTO sales (product_code, qty, price) VALUES (?, ?, ?)
};
I got it back as i wanted:
DBD::mysql::st execute failed: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'xINSERT INTO sample (product_code, qty, price) VALUES ('1', '2', '3')' at line 1
Sometimes it really helps. At least it did to me.
You can use DBI's ParamValues to obtain the parameter values but you are unlikely to find any method in a DBD to obtain the parameters actually in the SQL because they are mostly sent to the database after the SQL is parsed. You can look at DBIx::Log4perl to see how ParamValues is used in the error handler. You might also find some parts of DBIx::Log4perl useful.
There isn't a standard way to do that. The nearest approximation would be to substitute each placeholder by the (probably quoted) value. In general, it is fairly hard to reliably parse an SQL statement to find the question marks that are placeholders rather than parts of delimited identifiers or strings or comments. In the example, you can simply look for question marks; that won't always work, though:
INSERT /* ? */ INTO "??".Sales VALUES('?', ?, ?, ?);
Note that with most, but not necessarily all, DBMS (and hence most DBD drivers), the statement is sent to the DBMS when it is prepared; only the values are sent when the statement is executed. There never is a statement created with all the values substituted into the VALUES list, so neither DBI nor the DBMS should be expected to create one.
There isn't any generically supported DBI way to do this, I think, but individual database drivers might allow it. What database are you using?
A while back I was looking for a way to insert values into a text field through isql
and eventually found some load command that worked out for me.
It doesn't work when I try to execute it from Perl. I get a syntax error. I have tried two separate methods and both are not working so far.
I have the SQL statement variable print out at the end of each loop cycle so I know that the syntax is correct, but just not getting across correctly.
Here's the latest snip of code I was testing:
foreach(#files)
{
$STMT = <<EOF;
load from $_ insert into some_table
EOF
$sth = $db1->prepare($STMT);
$sth->execute;
}
#files is an array whose elements are a full path/location of a pipe-delimited text file (ex. /home/xx/xx/xx/something.txt)
The number of columns in the table match the number of fields in the text file and the type-checking is fine (I've loaded test files manually without fail)
The error I get back is:
DBD::Informix::db prepare failed: SQL: -201: A syntax error has occurred.
Any idea what might be causing this?
EDIT to RET's & Petr's answers
$STMT = "'LOAD FROM $_ INSERT INTO table'";
system("echo $STMT | isql $db")
I had to change it to this, because the die command would force an unnatural death and the statement had to be wrapped in single quotes.
Petr is exactly right, the LOAD statement is an ISQL or DB-Access extension, so you can't execute it through DBI. If you have a look at the manual, you'll see it is also invalid syntax for SPL, ESQL/C and so on.
It's not clear whether you have to use perl to execute the script, or perl is just a convenient way of generating the SQL.
If the former, and you want a pure-perl method, you have to prepare an INSERT statement (there's just one table involved by the look of it?), and slurp through the file, using split to break it up into columns and executing the prepared insert.
Otherwise, you can generate the SQL using perl and execute it through DB-Access, either directly with system or by wrapping both in either a shell script or DOS batch file.
System call version
foreach (#files) {
my $stmt = "LOAD FROM $_ INSERT INTO table;\n";
system("echo $stmt | dbaccess $database")
|| die "Statement $stmt failed: $!\n";
}
In a batch script version, you could write all the SQL into a single script, ie:
perl -e 'while(#ARGV){shift; print "LOAD FROM '$_' INSERT INTO table;\n"}' file1 [ file2 ... ] > loadfiles.sql
isql database loadfiles.sql
NB, the comment about quotes on the filename is only relevant if the filename contains spaces or metacharacters, the usual issue.
Also, one key difference in behaviour between isql and dbaccess is that when executed in this manner, dbaccess does not stop on error, but isql will. To make dbaccess stop processing on error, set DBACCNOIGN=1 in the environment.
Hope that's helpful.
This is because your query is not SQL query, it is an isql command that tells isql to parse the input file and generate INSERT statements.
If you think about it, the server can be on a completely different machine and has no idea what file are you talking about and how to access it.
So you basically have two options:
call isql and pipe the LOAD command to it - very ugly
parse the file yourself and generate the INSERT statements
Please note that the file Notes/load.unload is distributed with DBD::Informix and contains guidelines on how to handle UNLOAD operations using Perl, DBI and DBD::Informix. Somewhat to my chagrin, I see that it says "T.B.D." (more or less) for the LOAD section.
As other people have stated, the LOAD and UNLOAD statements are faked by various client-side tools to look like SQL statements, but the Informix server does not support them itself, mainly because of the issue with getting the file from a client machine (perhaps a PC) to the server machine (perhaps a Solaris machine).
To simulate the LOAD statement, you would need to analyze the INSERT INTO Table part. If it lists columns (INSERT INTO Table(Col03, Col05, Col09)), then you can expect three values in the load data file, and they go into those three columns. You would prepare a statement 'SELECT Col03, Col05, Col09 FROM Table' to get the types of the columns. Otherwise, you need to prepare a statement 'SELECT * FROM Table' to get the complete list of columns (and their types). Given the column names and the number of columns, you can create and prepare a suitable insert statement: 'INSERT INTO Table(Col03, Col05, Col09) VALUES(?,?,?)' or 'INSERT INTO Table VALUES(?,?,?,?,?,?,?,?,?)'. You could (arguably should) include column names in the second one.
With that ready, you now have parse the unloaded data. There is a document available in the SQLCMD program available from the IIUG Software Archive (which has been around a lot longer than Microsoft's upstart program of the same name). That describes the UNLOAD format in considerable detail. Perl has the ability to handle anything Informix uses - witness the UNLOAD information in the load.unload file distributed with DBD::Informix.
A quick bit of Googling showed that the syntax for load puts quote marks around the file name. What if you change your statement to be:
load from '$_' insert into some_table
Since your statement is not using place holders, you have to put the quotes in yourself, as opposed to using the DBI quoting functionality.