I'm trying to use SELECT regexp_replace(m.*, '[\n\r]+', ' ', 'g') to remove carriage returns and new lines from my field to generate a CSV from my table; however, looks like my postgresql version (7.4.27) does not support that function.
function regexp_replace(members, "unknown", "unknown", "unknown") does not exist
I also tried doing it this way:
SELECT replace(replace(m.*, '\r', ''), '\n', '')
function replace(members, "unknown", "unknown") does not exist
No function matches the given name and argument types. You may need to add explicit type casts.
or this way:
SELECT replace(replace(m.*, chr(13), ''), chr(10), '')
function replace(members, text, "unknown") does not exist
and still got similar errors.
How can a achieve that using another function or solution?
m.* makes no sense where you put it. It would work like this:
SELECT replace(replace(m.some_column, chr(13), ''), chr(10), '')
FROM tbl m;
But that just removes all "linefeed" and "carriage return" characters completely instead of replacing each string consisting only of these characters with a single space character like your original. If that's what you want, single character replacement is simpler and cheaper with translate() - also available in ancient pg 7.4:
SELECT translate(some_column, chr(13) || chr(10), '');
To achieve what your original regexp_replace() does (just without the nonsensical m.*), identify a single character that's not in the string and use that as stepping stone. Say: ° does not pop up, then:
SELECT replace(replace(replace(
translate(some_column, chr(13) || chr(10), '°') -- replace with dummy
, '°°', '°') -- consolidate to single dummy
, '°°', '°') -- repeat as many times as necessary
, '°', ' '); -- replace dummy with space
Looks awkward, and it's imperfect: fails for too many consecutive line breaks. But it's probably still faster than regexp_replace(), even in modern Postgres, because regular expressions are much more expensive. Then again, performance is probably not an issue here.
Upgrade to modern Postgres and you don't need this.
Related
I inserted a bunch of rows with a text field like content='...\n...\n...'.
I didn't use e in front, like conent=e'...\n...\n..., so now \n is not actually displayed as a newline - it's printed as text.
How do I fix this, i.e. how to change every row's content field from '...' to e'...'?
The syntax variant E'string' makes Postgres interpret the given string as Posix escape string. \n encoding a newline is only one of many interpreted escape sequences (even if the most common one). See:
Insert text with single quotes in PostgreSQL
To "re-evaluate" your Posix escape string, you could use a simple function with dynamic SQL like this:
CREATE OR REPLACE FUNCTION f_eval_posix_escapes(INOUT _string text)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE 'SELECT E''' || _string || '''' INTO _string;
END
$func$;
WARNING 1: This is inherently unsafe! We have to evaluate input strings dynamically without quoting and escaping, which allows SQL injection. Only use this in a safe environment.
WARNING 2: Don't apply repeatedly. Or it will misinterpret your actual string with genuine \ characters, etc.
WARNING 3: This simple function is imperfect as it cannot cope with nested single quotes properly. If you have some of those, consider instead:
Unescape a string with escaped newlines and carriage returns
Apply:
UPDATE tbl
SET content = f_eval_posix_escapes(content)
WHERE content IS DISTINCT FROM f_eval_posix_escapes(content);
db<>fiddle here
Note the added WHERE clause to skip updates that would not change anything. See:
How do I (or can I) SELECT DISTINCT on multiple columns?
Use REPLACE in an update query. Something like this: (I'm on mobile so please ignore any typo or syntax erro)
UPDATE table
SET
column = REPLACE(column, '\n', e'\n')
I have registration codes in my PostgreSQL table which are written messy, like MU-321-AB, MU/321/AB, MU 321-AB and so forth...
I would need to clear all of this to get MU321AB.
For this I uses following expression:
SELECT DISTINCT regexp_replace(ccode, '([^A-Za-z0-9])', ''), ...
This expression work as expected in 'NET' but not in PostgreSQL where it 'clears' only first occurrence of unwanted character.
How would I modify regular expression which will replace all unwanted chars in string to get clear code with only letters and numbers?
Use the global flag, but without any capture groups:
SELECT DISTINCT regexp_replace(ccode, '[^A-Za-z0-9]', '', 'g'), ...
Note that the global flag is part of the standard regular expression parser, so .NET is not following the standard in this case. Also, since you do not want anything extracted from the string - you just want to replace some characters - you should not use capture groups ().
I want to list the trigger no system ending with "_BI" in firebird database,
but no result with this
select * from rdb$triggers
where
rdb$trigger_source is not null
and (coalesce(rdb$system_flag,0) = 0)
and (rdb$trigger_source not starting with 'CHECK' )
and (rdb$trigger_name like '%BI')
but with this syntaxs it gives me a "_bi" and "_BI0U" and "_BI0U" ending result
and (rdb$trigger_name like '%BI%')
but with this syntaxs it gives me null result
and (rdb$trigger_name like '%#_BI')
thank you beforehand
The problem is that the Firebird system tables use CHAR(31) for object names, this means that they are padded with spaces up to the declared length. As a result, use of like '%BI') will not yield results, unless BI are the 30th and 31st character.
There are several solutions
For example you can trim the name before checking
trim(rdb$trigger_name) like '%BI'
or you can require that the name is followed by at least one space
rdb$trigger_name || ' ' like '%BI %'
On a related note, if you want to check if your trigger name ends in _BI, then you should also include the underscore in your condition. And as an underscore in like is a single character matcher, you need to escape it:
trim(rdb$trigger_name) like '%\_BI' escape '\'
Alternatively you could also try to use a regular expressions, as you won't need to trim or otherwise mangle the lefthand side of the expression:
rdb$trigger_name similar to '%\_BI[[:SPACE:]]*' escape '\'
I have a column url encoded with urlencode in php. I wish to make a select like this
SELECT some_mix_of_functions(...) AS Decoded FROM table
Replace is not a good solution because I will have to add all the decoding by hand. Any other solution to get the desire result ?
Yes you can:
CREATE OR REPLACE FUNCTION decode_url_part(p varchar) RETURNS varchar AS $$
SELECT convert_from(CAST(E'\\x' || string_agg(CASE WHEN length(r.m[1]) = 1 THEN encode(convert_to(r.m[1], 'SQL_ASCII'), 'hex') ELSE substring(r.m[1] from 2 for 2) END, '') AS bytea), 'UTF8')
FROM regexp_matches($1, '%[0-9a-f][0-9a-f]|.', 'gi') AS r(m);
$$ LANGUAGE SQL IMMUTABLE STRICT;
This creates a function decode_url_part, then you can use it like that:
SELECT decode_url_part('your%20urlencoded%20string')
Or you can just use the mix of functions and subqueries from the body of the above function.
This doesn't handle '+' characters (representing whitespace), but I guess adding this is quite easy (if you ever need it).
Also, this assumes utf-8 encoding for non-ascii characters, but you can replace 'UTF8' with your own encoding if you want.
It should be noted that the above code relies on undocumented postgresql feature, namely that the results of regexp_matches function are processed in the order they occur in the original string (which is natural, but not specified in docs).
As Pablo Santa Cruz notes, string_agg is a PostgreSQL 9.0 aggregate function. The equivalent code below doesn't use it (I hope it works for 8.x):
SELECT convert_from(CAST(E'\\x' || array_to_string(ARRAY(
SELECT CASE WHEN length(r.m[1]) = 1 THEN encode(convert_to(r.m[1], 'SQL_ASCII'), 'hex') ELSE substring(r.m[1] from 2 for 2) END
FROM regexp_matches($1, '%[0-9a-f][0-9a-f]|.', 'gi') AS r(m)
), '') AS bytea), 'UTF8');
Not out of the box. But you could create a pl/perl function that wraps the perl equivalent. (Or a pl/php function).
I have two strings in SQL and the REPLACE function only works on one of them, why is that?
Example 1:
SELECT REPLACE('18 286.74', ' ', '')
Example 2:
SELECT REPLACE('z z', ' ', '')
Example 1's output is still "18 286.74" whereas Example 2's output is "zz". Why does SQL not react the same way to both strings?
UPDATE:
When running select replace('123 123.12', ' ', '') that works fine, still not with '18 286.74'.
Test it the following way.
select unicode(substring('18 286.74', 3, 1))
If the code returns 32 then it's a space, if not, it's a different Unicode character and your replace ' ' won't work.
maybe cast is needed.
UPD: or not(on sql 2005 works fine too)
Are you sure it is a space? i.e. the same whitespace character that you are passing as the second argument? The code you've posted works fine for me on SQL Server 2008.
Re working on your friends PC - perhaps the whitespace got normalized when you sent it to him?
You are probably using non-breakable space.
I could reproduce it by typing ALT+0160 into the number in SELECT REPLACE('18 286.74', ' ', '')
Could you please issue this following:
SELECT CAST('18 286.74' AS BINARY), REPLACE('18 286.74', ' ', '')
by copying the '18 286.74' from REPLACE into CAST?
I was having the same issue and found that it was a char(10) (line feed). when copied out of Managment Studio it became a char(32) but in the record it was a char(10) try
Select Replace(#string, char(13), '')