I have a postgres function which runs the following loop
while x<=colnum LOOP
EXECUTE
'Update attendrpt set
slot'||x||' = pres
from (SELECT branch, semester, attend_date , div, array_to_string(ARRAY_AGG(first_name||':'||alias_name||':'||lect_type||':'||
to_char(present,'99')),';')
As pres
from attend1 where lecture_slot_no ='||x||'
group by branch, semester, attend_date , div ) j
where attendrpt.branch=j.branch and attendrpt.semester=j.semester
and attendrpt.attenddate=j.attend_date and attendrpt.div=j.div;';
`x:=x+1;
END LOOP;`
The problem here is it is conflicting the single quotes closing in query and the execute command.Is there anyway to solve this.
Thanks in advance.
Quote your function definition with dollar-quoting (like $BODY$ or just $$) as per the manual.
Use execute ... using instead of string substitution. For substituting identifiers use the %I format specifier from the format function.
If you absolutely must use || string concatenation, say if you're on some ancient version of PostgreSQL, you need to use the quote_literal and quote_ident functions to avoid issues with quoting and potential security problems.
Beyond that, it looks like the whole approach is completely unnecessary; you're doing something that looks like it can be done in simple SQL.
Related
I am trying to insert some text data into a table in SQL Server 9.
The text includes a single quote '.
How do I escape that?
I tried using two single quotes, but it threw me some errors.
eg. insert into my_table values('hi, my name''s tim.');
Single quotes are escaped by doubling them up, just as you've shown us in your example. The following SQL illustrates this functionality. I tested it on SQL Server 2008:
DECLARE #my_table TABLE (
[value] VARCHAR(200)
)
INSERT INTO #my_table VALUES ('hi, my name''s tim.')
SELECT * FROM #my_table
Results
value
==================
hi, my name's tim.
If escaping your single quote with another single quote isn't working for you (like it didn't for one of my recent REPLACE() queries), you can use SET QUOTED_IDENTIFIER OFF before your query, then SET QUOTED_IDENTIFIER ON after your query.
For example
SET QUOTED_IDENTIFIER OFF;
UPDATE TABLE SET NAME = REPLACE(NAME, "'S", "S");
SET QUOTED_IDENTIFIER ON;
-- set OFF then ON again
How about:
insert into my_table values('hi, my name' + char(39) + 's tim.')
Many of us know that the Popular Method of Escaping Single Quotes is by Doubling them up easily like below.
PRINT 'It''s me, Arul.';
we are going to look on some other alternate ways of escaping the single quotes.
1. UNICODE Characters
39 is the UNICODE character of Single Quote. So we can use it like below.
PRINT 'Hi,it'+CHAR(39)+'s Arul.';
PRINT 'Helo,it'+NCHAR(39)+'s Arul.';
2. QUOTED_IDENTIFIER
Another simple and best alternate solution is to use QUOTED_IDENTIFIER.
When QUOTED_IDENTIFIER is set to OFF, the strings can be enclosed in double quotes.
In this scenario, we don’t need to escape single quotes.
So,this way would be very helpful while using lot of string values with single quotes.
It will be very much helpful while using so many lines of INSERT/UPDATE scripts where column values having single quotes.
SET QUOTED_IDENTIFIER OFF;
PRINT "It's Arul."
SET QUOTED_IDENTIFIER ON;
CONCLUSION
The above mentioned methods are applicable to both AZURE and On Premises .
2 ways to work around this:
for ' you can simply double it in the string, e.g.
select 'I''m happpy' -- will get: I'm happy
For any charactor you are not sure of: in sql server you can get any char's unicode by select unicode(':') (you keep the number)
So this case you can also select 'I'+nchar(39)+'m happpy'
The doubling up of the quote should have worked, so it's peculiar that it didn't work for you; however, an alternative is using double quote characters, instead of single ones, around the string. I.e.,
insert into my_table values("hi, my name's tim.");
Also another thing to be careful of is whether or not it is really stored as a classic ASCII ' (ASCII 27) or Unicode 2019 (which looks similar, but not the same). This isn't a big deal on inserts, but it can mean the world on selects and updates. If it's the unicode value then escaping the ' in a WHERE clause (e.g where blah = 'Workers''s Comp') will return like the value you are searching for isn't there if the ' in "Worker's Comp" is actually the unicode value.If your client application supports free-key, as well as copy and paste based input, it could be Unicode in some rows, and ASCII in others!
A simple way to confirm this is by doing some kind of open ended query that will bring back the value you are searching for, and then copy and paste that into notepad++ or some other unicode supporting editor. The differing appearance between the ascii value and the unicode one should be obvious to the eyes, but if you lean towards the anal, it will show up as 27 (ascii) or 92 (unicode) in a hex editor.
The following syntax will escape you ONLY ONE quotation mark:
SELECT ''''
The result will be a single quote. Might be very helpful for creating dynamic SQL :).
Double quotes option helped me
SET QUOTED_IDENTIFIER OFF;
insert into my_table values("hi, my name's tim.");
SET QUOTED_IDENTIFIER ON;
This should work
DECLARE #singleQuote CHAR
SET #singleQuote = CHAR(39)
insert into my_table values('hi, my name'+ #singleQuote +'s tim.')
Just insert a ' before anything to be inserted. It will be like a escape character in sqlServer
Example:
When you have a field as, I'm fine.
you can do:
UPDATE my_table SET row ='I''m fine.';
I had the same problem, but mine was not based of static data in the SQL code itself, but from values in the data.
This code lists all the columns names and data types in my database:
SELECT DISTINCT QUOTENAME(COLUMN_NAME),DATA_TYPE FROM INFORMATION_SCHEMA.COLUMNS
But some column names actually have a single-quote embedded in the name of the column!, such as ...
[MyTable].[LEOS'DATACOLUMN]
To process these, I had to use the REPLACE function along with the suggested QUOTED_IDENTIFIER setting. Otherwise it would be a syntax error, when the column is used in a dynamic SQL.
SET QUOTED_IDENTIFIER OFF;
SET #sql = 'SELECT DISTINCT ''' + #TableName + ''',''' + REPLACE(#ColumnName,"'","''") + ...etc
SET QUOTED_IDENTIFIER ON;
The STRING_ESCAPE funtion can be used on newer versions of SQL Server
This should work: use a back slash and put a double quote
"UPDATE my_table SET row =\"hi, my name's tim.\";
I inserted a bunch of rows with a text field like content='...\n...\n...'.
I didn't use e in front, like conent=e'...\n...\n..., so now \n is not actually displayed as a newline - it's printed as text.
How do I fix this, i.e. how to change every row's content field from '...' to e'...'?
The syntax variant E'string' makes Postgres interpret the given string as Posix escape string. \n encoding a newline is only one of many interpreted escape sequences (even if the most common one). See:
Insert text with single quotes in PostgreSQL
To "re-evaluate" your Posix escape string, you could use a simple function with dynamic SQL like this:
CREATE OR REPLACE FUNCTION f_eval_posix_escapes(INOUT _string text)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE 'SELECT E''' || _string || '''' INTO _string;
END
$func$;
WARNING 1: This is inherently unsafe! We have to evaluate input strings dynamically without quoting and escaping, which allows SQL injection. Only use this in a safe environment.
WARNING 2: Don't apply repeatedly. Or it will misinterpret your actual string with genuine \ characters, etc.
WARNING 3: This simple function is imperfect as it cannot cope with nested single quotes properly. If you have some of those, consider instead:
Unescape a string with escaped newlines and carriage returns
Apply:
UPDATE tbl
SET content = f_eval_posix_escapes(content)
WHERE content IS DISTINCT FROM f_eval_posix_escapes(content);
db<>fiddle here
Note the added WHERE clause to skip updates that would not change anything. See:
How do I (or can I) SELECT DISTINCT on multiple columns?
Use REPLACE in an update query. Something like this: (I'm on mobile so please ignore any typo or syntax erro)
UPDATE table
SET
column = REPLACE(column, '\n', e'\n')
I have a table test(id,name).
I need to insert values like: user's log, 'my user', customer's.
insert into test values (1,'user's log');
insert into test values (2,''my users'');
insert into test values (3,'customer's');
I am getting an error if I run any of the above statements.
If there is any method to do this correctly please share. I don't want any prepared statements.
Is it possible using sql escaping mechanism?
String literals
Escaping single quotes ' by doubling them up → '' is the standard way and works of course:
'user's log' -- incorrect syntax (unbalanced quote)
'user''s log'
Plain single quotes (ASCII / UTF-8 code 39), mind you, not backticks `, which have no special purpose in Postgres (unlike certain other RDBMS) and not double-quotes ", used for identifiers.
In old versions or if you still run with standard_conforming_strings = off or, generally, if you prepend your string with E to declare Posix escape string syntax, you can also escape with the backslash \:
E'user\'s log'
Backslash itself is escaped with another backslash. But that's generally not preferable.
If you have to deal with many single quotes or multiple layers of escaping, you can avoid quoting hell in PostgreSQL with dollar-quoted strings:
'escape '' with '''''
$$escape ' with ''$$
To further avoid confusion among dollar-quotes, add a unique token to each pair:
$token$escape ' with ''$token$
Which can be nested any number of levels:
$token2$Inner string: $token1$escape ' with ''$token1$ is nested$token2$
Pay attention if the $ character should have special meaning in your client software. You may have to escape it in addition. This is not the case with standard PostgreSQL clients like psql or pgAdmin.
That is all very useful for writing PL/pgSQL functions or ad-hoc SQL commands. It cannot alleviate the need to use prepared statements or some other method to safeguard against SQL injection in your application when user input is possible, though. #Craig's answer has more on that. More details:
SQL injection in Postgres functions vs prepared queries
Values inside Postgres
When dealing with values inside the database, there are a couple of useful functions to quote strings properly:
quote_literal() or quote_nullable() - the latter outputs the unquoted string NULL for null input.
There is also quote_ident() to double-quote strings where needed to get valid SQL identifiers.
format() with the format specifier %L is equivalent to quote_nullable().
Like: format('%L', string_var)
concat() or concat_ws() are typically no good for this purpose as those do not escape nested single quotes and backslashes.
According to PostgreSQL documentation (4.1.2.1. String Constants):
To include a single-quote character within a string constant, write
two adjacent single quotes, e.g. 'Dianne''s horse'.
See also the standard_conforming_strings parameter, which controls whether escaping with backslashes works.
This is so many worlds of bad, because your question implies that you probably have gaping SQL injection holes in your application.
You should be using parameterized statements. For Java, use PreparedStatement with placeholders. You say you don't want to use parameterised statements, but you don't explain why, and frankly it has to be a very good reason not to use them because they're the simplest, safest way to fix the problem you are trying to solve.
See Preventing SQL Injection in Java. Don't be Bobby's next victim.
There is no public function in PgJDBC for string quoting and escaping. That's partly because it might make it seem like a good idea.
There are built-in quoting functions quote_literal and quote_ident in PostgreSQL, but they are for PL/PgSQL functions that use EXECUTE. These days quote_literal is mostly obsoleted by EXECUTE ... USING, which is the parameterised version, because it's safer and easier. You cannot use them for the purpose you explain here, because they're server-side functions.
Imagine what happens if you get the value ');DROP SCHEMA public;-- from a malicious user. You'd produce:
insert into test values (1,'');DROP SCHEMA public;--');
which breaks down to two statements and a comment that gets ignored:
insert into test values (1,'');
DROP SCHEMA public;
--');
Whoops, there goes your database.
In postgresql if you want to insert values with ' in it then for this you have to give extra '
insert into test values (1,'user''s log');
insert into test values (2,'''my users''');
insert into test values (3,'customer''s');
you can use the postrgesql chr(int) function:
insert into test values (2,'|| chr(39)||'my users'||chr(39)||');
When I used Python to insert values into PostgreSQL, I also met the question: column "xxx" does not exist.
The I find the reason in wiki.postgresql:
PostgreSQL uses only single quotes for this (i.e. WHERE name = 'John'). Double quotes are used to quote system identifiers; field names, table names, etc. (i.e. WHERE "last name" = 'Smith').
MySQL uses ` (accent mark or backtick) to quote system identifiers, which is decidedly non-standard.
It means PostgreSQL can use only single quote for field names, table names, etc. So you can not use single quote in value.
My situation is: I want to insert values "the difference of it’s adj for sb and it's adj of sb" into PostgreSQL.
How I figure out this problem:
I replace ' with ’, and I replace " with '. Because PostgreSQL value does not support double quote.
So I think you can use following codes to insert values:
insert into test values (1,'user’s log');
insert into test values (2,'my users');
insert into test values (3,'customer’s');
If you need to get the work done inside Pg:
to_json(value)
https://www.postgresql.org/docs/9.3/static/functions-json.html#FUNCTIONS-JSON-TABLE
You must have to add an extra single quotes -> ' and make doubling quote them up like below examples -> ' ' is the standard way and works of course:
Wrong way: 'user's log'
Right way: 'user''s log'
problem:
insert into test values (1,'user's log');
insert into test values (2,''my users'');
insert into test values (3,'customer's');
Solutions:
insert into test values (1,'user''s log');
insert into test values (2,'''my users''');
insert into test values (3,'customer''s');
I need to do an UPDATE script using the Replace() function of Postgres but I don't know the exact string that I have to replace and I'd like to know if there is some way that I can do this similary the LIKEoperator, using Wildcards.
My problem is that I got a table that contains some scripts and at the end of each one there is a tag <signature> like this:
'SELECT SCRIPT WHATEVER.... < signature>782798e2a92c72b270t920b< signature>'
What I need to do is:
UPDATE table SET script = REPLACE(script,'<signature>%<signature>','<signature>1234ABCDEF567890<signature>')
Whatever the signature is, I need to replace with a new one defined by me. I know using the '%' doesn't work, it was just to ilustrate the effect i want to perform. Is there any way to do this in Postgres 9.5?
with expr
as
(select 'Hello <signature>thisismysig</signature>'::text as
full_text, '<signature>'::text as open,
'</signature>'::text as close
)
select
substring(full_text from
position(open in full_text)+char_length(open)
for
position(close in full_text)- char_length(open)-position(open in full_text)
)
note: with part added for ease of understanding (hopefully).
Use POSIX regex to do the same thing as other answer (but shorter)
select
substring('a bunch of other stuff <signature>mysig</signature>'
from '<signature>(.*?)</signature>')
I have a column url encoded with urlencode in php. I wish to make a select like this
SELECT some_mix_of_functions(...) AS Decoded FROM table
Replace is not a good solution because I will have to add all the decoding by hand. Any other solution to get the desire result ?
Yes you can:
CREATE OR REPLACE FUNCTION decode_url_part(p varchar) RETURNS varchar AS $$
SELECT convert_from(CAST(E'\\x' || string_agg(CASE WHEN length(r.m[1]) = 1 THEN encode(convert_to(r.m[1], 'SQL_ASCII'), 'hex') ELSE substring(r.m[1] from 2 for 2) END, '') AS bytea), 'UTF8')
FROM regexp_matches($1, '%[0-9a-f][0-9a-f]|.', 'gi') AS r(m);
$$ LANGUAGE SQL IMMUTABLE STRICT;
This creates a function decode_url_part, then you can use it like that:
SELECT decode_url_part('your%20urlencoded%20string')
Or you can just use the mix of functions and subqueries from the body of the above function.
This doesn't handle '+' characters (representing whitespace), but I guess adding this is quite easy (if you ever need it).
Also, this assumes utf-8 encoding for non-ascii characters, but you can replace 'UTF8' with your own encoding if you want.
It should be noted that the above code relies on undocumented postgresql feature, namely that the results of regexp_matches function are processed in the order they occur in the original string (which is natural, but not specified in docs).
As Pablo Santa Cruz notes, string_agg is a PostgreSQL 9.0 aggregate function. The equivalent code below doesn't use it (I hope it works for 8.x):
SELECT convert_from(CAST(E'\\x' || array_to_string(ARRAY(
SELECT CASE WHEN length(r.m[1]) = 1 THEN encode(convert_to(r.m[1], 'SQL_ASCII'), 'hex') ELSE substring(r.m[1] from 2 for 2) END
FROM regexp_matches($1, '%[0-9a-f][0-9a-f]|.', 'gi') AS r(m)
), '') AS bytea), 'UTF8');
Not out of the box. But you could create a pl/perl function that wraps the perl equivalent. (Or a pl/php function).