I may have an odd request. I'm not finding any help via Google.
I am using the DbVisualizer Pro 10.0.15 gui tool connected to a PostgreSQL db.
I need to create a csv file from a database table. I select the records I need in a query then export the results to a .csv file. I can do that easy.
select note from notes;
highlight all results records >> right-click >> select export >> choose csv
Some of the records have both single and/or double-quotes in the content.
The person receiving this file needs to upload the csv file into another system. They are stating that these single and double-quotes in the content will not work in their upload. I've been asked to escape these quotes. They want to keep them in the content, but have them appear in the field with the backslash escape character, i.e: it is John's ball would show in the csv file as: it is John\'s ball. The same for dbl-quotes.
I could probably do this with a search-and-replace function in a text editor after creating the csv file, but I'd like to think this can be done via sql.
I've tried playing with the regexp_replace() function.
select regexp_replace(note, '"', '\"') as notes from notes works on the dbl-quotes, but I'm not having any luck on the single quotes.
Help? Is there a way to do this?
You can escape double quotes by doing:
postgres=# SELECT REGEXP_REPLACE('this "is" a string', '"', '\"', 'g');
regexp_replace
----------------------
this \"is\" a string
(1 row)
For single quotes, the approach is similar, but you have to escape them using another single quote. So instead of having something like /', it should be ''. The query is:
postgres=# SELECT REGEXP_REPLACE('this ''is'' a string', '''', '\''', 'g');
regexp_replace
----------------------
this \'is\' a string
(1 row)
Note the 'g' flag in the end, this forces it to replace all occurrences and not just the first one found.
You can also replace both single and double quotes in a single statement, although they are replaced with the same string (\" in this case).
postgres=# SELECT REGEXP_REPLACE('this "is" a ''normal'' string', '["'']', '\"', 'g');
regexp_replace
---------------------------------
this \"is\" a \"normal\" string
(1 row)
Related
I inserted a bunch of rows with a text field like content='...\n...\n...'.
I didn't use e in front, like conent=e'...\n...\n..., so now \n is not actually displayed as a newline - it's printed as text.
How do I fix this, i.e. how to change every row's content field from '...' to e'...'?
The syntax variant E'string' makes Postgres interpret the given string as Posix escape string. \n encoding a newline is only one of many interpreted escape sequences (even if the most common one). See:
Insert text with single quotes in PostgreSQL
To "re-evaluate" your Posix escape string, you could use a simple function with dynamic SQL like this:
CREATE OR REPLACE FUNCTION f_eval_posix_escapes(INOUT _string text)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE 'SELECT E''' || _string || '''' INTO _string;
END
$func$;
WARNING 1: This is inherently unsafe! We have to evaluate input strings dynamically without quoting and escaping, which allows SQL injection. Only use this in a safe environment.
WARNING 2: Don't apply repeatedly. Or it will misinterpret your actual string with genuine \ characters, etc.
WARNING 3: This simple function is imperfect as it cannot cope with nested single quotes properly. If you have some of those, consider instead:
Unescape a string with escaped newlines and carriage returns
Apply:
UPDATE tbl
SET content = f_eval_posix_escapes(content)
WHERE content IS DISTINCT FROM f_eval_posix_escapes(content);
db<>fiddle here
Note the added WHERE clause to skip updates that would not change anything. See:
How do I (or can I) SELECT DISTINCT on multiple columns?
Use REPLACE in an update query. Something like this: (I'm on mobile so please ignore any typo or syntax erro)
UPDATE table
SET
column = REPLACE(column, '\n', e'\n')
I am trying to remove using REGEXP_REPLACE the following special characters: "[]{}
from the following text field: [{"x":"y","s":"G_1","cn":"C8"},{"cn":"M2","gn":"G_2","cn":"CA99"},{"c":"ME3","gn":"G_3","c":"CA00"}]
and replace them with nothing, not even a space.
*Needless to say, this is just an example string, and I need to find a consistent solution for similar but different strings.
I was trying to run the following: SELECT REGEXP_REPLACE('[{"x":"y","s":"G_1","cn":"C8"},{"cn":"M2","gn":"G_2","cn":"CA99"},{"c":"ME3","gn":"G_3","c":"CA00"}] ','[{[}]":]','')
But received pretty much the same string..
Thanks in advance!
You need to escape the special characters (\), and to specify that you want to repeat the operation for every characters ('g') else it will stop at the 1st match
SELECT REGEXP_REPLACE(
'[{"x":"y","s":"G_1","cn":"C8"},{"cn":"M2","gn":"G_2","cn":"CA99"},{"c":"ME3","gn":"G_3","c":"CA00"}] ',
'[{\[}\]":]',
'',
'g');
regexp_replace
--------------------------------------------------
xy,sG_1,cnC8,cnM2,gnG_2,cnCA99,cME3,gnG_3,cCA00
(1 row)
Long story...
I am trying to geenrate a crosstab query dynamically and run it as a psql script..
To achieve this, I want the last line of the sql to generated and appended to the top portion of the sql.
The last line of the sql is like this.... "as final_result(symbol character varying,"431" numeric,"432" numeric,"433" numeric);"
Of which, the "431", "432" etc are to be generated dynamically as these are the pivot columns and they change from time to time...
So I wrote a query to output the text as follows....
psql -c "select distinct '"'||runweek||'" numeric ,' from calendar where runweek between current_runweek()-2 and current_runweek() order by 1;" -U USER -d DBNAME > /tmp/gengen.lst
While the sql provides the output, when I run it as a script, because of the special characters (', "", ) it fails.
How should I get it working? My plan was then loop through the "lst" file and build the pivot string, and append that to the top portion of the sql and execute the script... (New to postgres, does not know , dynamic sql generation and execution etc.. but I am comfortable with UNIX scripting..)
If I could somehow get the output as
("431" numeric, "432" numeric....etc) in a single step, if there is a recommendation to achieve this, it will be greatly appreciated.....
Since you're using double quotes around the argument, double quotes inside the argument must be escaped with a backslash:
psql -c "select distinct '\"'||runweek||'\" numeric ,' from calendar where runweek between current_runweek()-2 and current_runweek() order by 1;"
Heredoc can also be used instead of -c. It accepts multi-line formatting so that makes the whole thing more readable.
(psql [arguments] <<EOF
select distinct '"'||runweek||'" numeric ,'
from calendar
where runweek between current_runweek()-2 and current_runweek()
order by 1;
EOF
) > output
By using quote_ident which is specifically meant to produce a quoted identifier from a text value, you don't even need to add the double quotes. The query could be like:
select string_agg( quote_ident(runweek::text), ',' order by runweek)
from calendar
where runweek between current_runweek()-2 and current_runweek();
which also solves the problem that your original query has a stray ',' at the end, whereas this form does not.
I tried to select a data which is in column "fileName" and its fileName is '2016-11-22-12-55-09_hyun.png'
I tired the
select * from images where 'fileName' like '2016-11-22-12-55-09_hyun.png'
However it can not select anything, nor has any kind of error info.
How can I select this file with its filename? Thank you so much.
Single quotes denote a string literal. So in this query you aren't evaluating the column filename, but checking whether the string 'filename' is like the string '2016-11-22-12-55-09_hyun.png', which it of course is not. Just drop the quotes from filename and you should be OK. Also note that since you aren't using any wildcards, using the like operator is pretty pointless, and you could (should) just a plain old equality check:
select * from images where fileName = '2016-11-22-12-55-09_hyun.png'
-- No quotes -------------^--------^
I have a query:
SELECT phone,
to_char(appt_date,'MM/DD/YYYY'),
time_text,
staff_email || subject_line as title,
staff_wav,
agency_wav
FROM phone_dialer
that is sent to a csv file
That results in
9105554444,07/01/2011,08:30am,me#myemail.orgGADK082493,staffid0827,Coastal Counseling
or
9105554444,07/01/2011,08:30am,me#myemail.orgGADK082493,staffid0827,Smith, Bob
The "agency_wav" column could have a name of company. I have tried several ways to remove the comma between Smith, Bob and am failing miserably.
Can anyone steer me to a resolution?
Answer to title, since the body of the question is unclear.
Fastest way to remove commas from a string:
SELECT translate('Smith, Bob, foo,,,bar', ',', '');
Related answer addressing translate() / replace():
Looking for phone number containing a minus, like "123-456789"
If your surround your query with the syntax COPY () TO STDOUT WITH CSV; then it will construct the CSV output and automatically quote the field values that contain commas.
If you want to manually do it in the query, try replace(agency_wav,',','').
The preferred way to create CSV is to use COPY command.
If by some reason you don't want or can't use it, you just need make value returned in the column CSV friendly that is enclose value in double quotes and escape existing double quotes by duplicating them in the string. This will preserve correct value (that is all commas) but will not break CSV format.
SELECT phone,
to_char(appt_date,'MM/DD/YYYY'),
time_text,
staff_email || subject_line as title,
staff_wav,
'"' || replace(agency_wav, '"', '""') || '"'
FROM phone_dialer
This will produce the following line
9105554444,07/01/2011,08:30am,me#myemail.orgGADK082493,staffid0827,"Smith, Bob"
Note quoted value which has comma.