I am importing data from a table which has raw feeds in Varchar, I need to import a column in varchar into a string column. I tried using the <column_name>::integer as well as to_number(<column_name>,'9999999') but I am getting errors, as there are a few empty fields, I need to retrieve them as empty or null into the new table.
Wild guess: If your value is an empty string, you can use NULLIF to replace it for a NULL:
SELECT
NULLIF(your_value, '')::int
You can even go one further and restrict on this coalesced field such as, for example:-
SELECT CAST(coalesce(<column>, '0') AS integer) as new_field
from <table>
where CAST(coalesce(<column>, '0') AS integer) >= 10;
If you need to treat empty columns as NULLs, try this:
SELECT CAST(nullif(<column>, '') AS integer);
On the other hand, if you do have NULL values that you need to avoid, try:
SELECT CAST(coalesce(<column>, '0') AS integer);
I do agree, error message would help a lot.
The only way I succeed to not having an error because of NULL, or special characters or empty string is by doing this:
SELECT REGEXP_REPLACE(COALESCE(<column>::character varying, '0'), '[^0-9]*' ,'0')::integer FROM table
I'm not able to comment (too little reputation? I'm pretty new) on Lukas' post.
On my PG setup to_number(NULL) does not work, so my solution would be:
SELECT CASE WHEN column = NULL THEN NULL ELSE column :: Integer END
FROM table
If the value contains non-numeric characters, you can convert the value to an integer as follows:
SELECT CASE WHEN <column>~E'^\\d+$' THEN CAST (<column> AS INTEGER) ELSE 0 END FROM table;
The CASE operator checks the < column>, if it matches the integer pattern, it converts the rate into an integer, otherwise it returns 0
Common issue
Naively type casting any string into an integer like so
SELECT ''::integer
Often results to the famous error:
Query failed: ERROR: invalid input syntax for integer: ""
Problem
PostgreSQL has no pre-defined function for safely type casting any string into an integer.
Solution
Create a user-defined function inspired by PHP's intval() function.
CREATE FUNCTION intval(character varying) RETURNS integer AS $$
SELECT
CASE
WHEN length(btrim(regexp_replace($1, '[^0-9]', '','g')))>0 THEN btrim(regexp_replace($1, '[^0-9]', '','g'))::integer
ELSE 0
END AS intval;
$$
LANGUAGE SQL
IMMUTABLE
RETURNS NULL ON NULL INPUT;
Usage
/* Example 1 */
SELECT intval('9000');
-- output: 9000
/* Example 2 */
SELECT intval('9gag');
-- output: 9
/* Example 3 */
SELECT intval('the quick brown fox jumps over the lazy dog');
-- output: 0
you can use this query
SUM(NULLIF(conversion_units, '')::numeric)
And if your column has decimal points
select NULLIF('105.0', '')::decimal
This works for me:
select (left(regexp_replace(coalesce('<column_name>', '0') || '', '[^0-9]', '', 'g'), 8) || '0')::integer
For easy view:
select (
left(
regexp_replace(
-- if null then '0', and convert to string for regexp
coalesce('<column_name>', '0') || '',
'[^0-9]',
'',
'g'
), -- remove everything except numbers
8 -- ensure ::integer doesn't overload
) || '0' -- ensure not empty string gets to ::integer
)::integer
The perfect solution for me is to use nullif and regexp_replace
SELECT NULLIF(REGEXP_REPLACE('98123162t3712t37', '[^0-9]', '', 'g'), '')::bigint;
Above solution consider the following edge cases.
String and Number: only the regexp_replace function perfectly converts into integers.
SELECT NULLIF(REGEXP_REPLACE('string and 12345', '[^0-9]', '', 'g'), '')::bigint;
Only string: regexp_replace converts non-string characters to empty strings; which can't cast directly to integer so use nullif to convert to null
SELECT NULLIF(REGEXP_REPLACE('only string', '[^0-9]', '', 'g'), '')::bigint;
Integer range: Converting a string into integer may cause out of range for type integer error. So use bigint instead
SELECT NULLIF(REGEXP_REPLACE('98123162t3712t37', '[^0-9]', '', 'g'), '')::bigint;
Related
I am having a hard time about this.
I am trying to cast a varchar containing a list of numbers into an int array, in order to serve an in operator on a where clause.
This is the last version of my code.
create or replace function is_product_in_categories (
_product_id integer,
_cat_list varchar
)
returns boolean
as $$
declare
_n integer;
begin
_n = 0;
select count(*)
into _n
from category_products
where product_id = _product_id and category_id in (_cat_list::int[]);
return _n > 0;
end;
$$ language plpgsql;
select is_product_in_categories(1, '1, 2, 3');
Error is
SQL Error [42883]: ERROR: operator does not exist: integer = integer[]
Hint: No operator matches the given name and argument types. You might need to add explicit type casts.
Where: PL/pgSQL function is_product_in_categories(integer,character varying) line 7 at SQL statement
I have tried several arguments, as '1, 2, 3', '(1, 2, 3)' or '[1, 2, 3]'. Also removing parenthesis near the in operator, etc.
Any idea?
Use string_to_array() to convert a string to a (text) array:
SELECT string_to_array('1, 2, 3', ', ')::int[]; -- use ::int[] to cast to an int array
+---------------+
|string_to_array|
+---------------+
|{1,2,3} |
+---------------+
If you control the string (e.g you are constructing it yourself), you can use either of those two:
SELECT ARRAY[1, 2, 3] -- no need to cast this one
, '{1, 2, 3}'::int[]; -- you have to specify that it's an array, not simply a string value
+-------+-------+
|array |int4 |
+-------+-------+
|{1,2,3}|{1,2,3}|
+-------+-------+
The problem with the in operator is it doesn't admit an array as an argument. Instead it expects a simple list of scalars. See PostgreSQL documentation here https://www.postgresql.org/docs/9.0/functions-comparisons.html#AEN16964
To avoid this limitation the = any combination accepts an array as an argument.
The code ends this way.
create or replace function is_product_in_categories (
_product_id integer,
_cat_list varchar
)
returns boolean
as $$
declare
_n integer;
begin
_n = 0;
select count(*)
into _n
from of_category_products
where product_id = _product_id and category_id = any (_cat_list::int[]);
return _n > 0;
end;
$$ language plpgsql;
select is_product_in_categories(1, '{1, 2, 3}')
Also, the syntax for literal arrays, using {} has been observed, following Bergi comment.
Revise your function declaration and define as variadic integer array:
create or replace function is_product_in_categories (
_product_id integer,
Variadic _cat_list integer[] )
or just as a array of integers:
create or replace function is_product_in_categories (
_product_id integer,
_cat_list integer[] )
Either way you can reduce the function to a single statement.
create or replace function is_product_in_categories3 (
_product_id integer,
_cat_list integer[]
)
returns boolean
language sql
as $$
select
exists (select null
from category_products
where product_id = _product_id and category_id = any(_cat_list)
);
$$;
See here for full example of both.
I have two table, with two colounms:
Tab1.Number1 (BigInt)
100000
100001
tab2.Number2 (character varying)
10000_300
10001_301
90009_222
I need to do a select that show the rows which have the same values (without the '_').
i tried to convert the value with
to_number(tab2.Number2, 99999)
But it doesn't work:
to_number(character varying, integer) does not exist.
Database is postgress.
Can you help me?
try to replace the character '_' to '' before cast it to number
example:
cast(replace(tab2.Number2, '_', '') as int)
for example in a table one field is character varying so how to convert that character varying to an Integer type, what exactly is I need to get the return value in Integer datatype
Database : postgresql-9.2
i would suggest to make Functions to do the job i.e
CREATE OR REPLACE FUNCTION chartoint(charparam character varying)
RETURNS integer AS
$BODY$
SELECT CASE WHEN trim($1) ~ '[0-9]+' THEN CAST(trim($1) AS integer) ELSE NULL END;
$BODY$
LANGUAGE sql IMMUTABLE STRICT
You may try like this:
SELECT NULLIF(your_value, '')::int
or extend that like thats
SELECT CAST(coalesce(column_name, '0') AS integer) as Value
from table_name
SELECT mycolumn::integer FROM mytable WHERE something
SELECT CASE WHEN mycolumn = NULL THEN NULL ELSE mycolumn :: Integer END
FROM mytable WHERE something
I am trying check if a value is null if so the select null else cast to numeric, but it throws an error. This is actually part of an insert statement
INSERT into someTable(name,created,power)
SELECT 'xyz',now(),
case when :power ='null' then NULL else cast(:power as numeric) end from abc
error that I get is
Error: ERROR: invalid input syntax for type numeric: "null"
:power is a variable that can be given any value using java code. If I give a value of null it give an error.
In code I get the following error from the java stack trace
org.postgresql.util.PSQLException: ERROR: cannot cast type bytea to numeric
Error:
SELECT CASE WHEN 'null' = 'null' THEN NULL ELSE cast('null' AS numeric) END
No error:
DO $$
DECLARE
power text := 'null';
BEGIN
PERFORM CASE WHEN power = 'null' THEN NULL ELSE cast(power AS numeric) END;
END;
$$
Explanation:
If you build a query string, the expression cast('null' AS numeric) or simply 'null'::numeric always raises an exception, even in an ELSE block that is never executed, because it is invalid input syntax and the exception is raised during the syntax check (like the error message implies), not during execution.
A CASE statement like you display only makes sense with a parameter or variable not with literals. The second instance of the literal has no connection to the first instance whatsoever after the query string has been assembled.
For dynamic SQL like that, you need to check the value before you build the query string. Or you use a function or prepared statement and pass the value as parameter. That would work, too.
More advice after comment:
In your particular case you could check the value in the app and build a query string like this:
INSERT INTO tbl(name, abc_id, created, power)
SELECT 'xyz'
, abc_id
, now()
, <insert_value_of_power_or_NULL_here> -- automatically converted to numeric
FROM abc
You may be interested in a different approach to INSERT data from a file conditionally.
Use COPY for files local to the server or psql's meta-command \copy for files local to the client.
if the field value is null, and you want in this case to map it to some value you can use coalesce(field_name, 'Some value') or coalesce(field_name, 123).
For full documentation see here.
You have to check with the IS operator, and not with the equal when you dealing with NULL :
INSERT into someTable(name,created,power)
SELECT 'xyz',now(),
case when :power IS null then NULL else cast(:power as numeric) end from abc
INSERT into someTable(name,created,power) SELECT 'xyz',now(),
case :power when 'null' then NULL else :power end::numeric from abc
I was trying to do something similar in order to update/insert some records where a numeric value can be null or not.
You can validate a variable before you send it to the function or inside the function depending the value passed
(For me using a variable is better than use CASE WHEN THEN ELSE END CASE every time you need to validate the value)
So to work with the NULL values using a regular comparison operand in order to find a record to update can be done by turning transform_null_equals to ON
I hope this help someone
CREATE OR REPLACE FUNCTION update_insert_transaction(vcodaccount integer, vcodaccountaux text,
vdescription text, vcodgroup integer)
RETURNS integer AS $$
DECLARE
n integer = 0;
vsql text = 'NULL';
BEGIN
IF vcodaccountaux <> '' THEN
vsql = vcodaccountaux;
END IF;
SET LOCAL transform_null_equals TO ON;
EXECUTE 'UPDATE account_import_conf SET (codaccount, codaccountaux, description, codgroup) =
('||vcodaccount||','||vsql||',trim('||quote_literal(vdescription)||'),'||vcodgroup||')
WHERE codaccount='||vcodaccount||' AND codaccountaux = '||vsql||' RETURNING * ';
GET DIAGNOSTICS n = ROW_COUNT;
IF n = 0 THEN
EXECUTE 'INSERT INTO account_import_conf (codaccount, codaccountaux, description, codgroup)
SELECT '||vcodaccount||','||vsql||' ,trim('||quote_literal(vdescription)||'),'||vcodgroup||';';
END IF;
RETURN n;
END;$$
LANGUAGE plpgsql;
I want to find the first and the last occurrences of a specific character inside a string. As an example, consider a string named "2010-####-3434", and suppose the character to be searched for is "#". The first occurrence of hash inside the string is at 6-th position, and the last occurrence is at 9-th position.
Well...
Select position('#' in '2010-####-3434');
will give you the first.
If you want the last, just run that again with the reverse of your string. A pl/pgsql string reverse can be found here.
Select length('2010-####-3434') - position('#' in reverse_string('2010-####-3434')) + 1;
My example:
reverse(substr(reverse(newvalue),0,strpos(reverse(newvalue),',')))
Reverse all string
Substring string
Reverse result
In the case where char = '.', an escape is needed. So the function can be written:
CREATE OR REPLACE FUNCTION last_post(text,char)
RETURNS integer LANGUAGE SQL AS $$
select length($1)- length(regexp_replace($1, E'.*\\' || $2,''));
$$;
9.5+ with array_positions
Using basic PostgreSQL array functions we call string_to_array(), and then feed that to array_positions() like this array_positions(string_to_array(str,null), c)
SELECT
arrpos[array_lower(arrpos,1)] AS first,
arrpos[array_upper(arrpos,1)] AS last
FROM ( VALUES
('2010-####-3434', '#')
) AS t(str,c)
CROSS JOIN LATERAL array_positions(string_to_array(str,null), c)
AS arrpos;
I do not know how to do that, but the regular expression functions like regexp_matches, regexp_replace, and regexp_split_to_array may be an alternative route to solving your problem.
This pure SQL function will provide the last position of a char inside the string, counting from 1. It returns 0 if not found ... But (big disclaimer) it breaks if the character is some regex metacharacter ( .$^()[]*+ )
CREATE FUNCTION last_post(text,char) RETURNS integer AS $$
select length($1)- length(regexp_replace($1, '.*' || $2,''));
$$ LANGUAGE SQL IMMUTABLE;
test=# select last_post('hi#-#-#byte','#');
last_post
-----------
7
test=# select last_post('hi#-#-#byte','a');
last_post
-----------
0
A more robust solution would involve pl/pgSQL, as rfusca's answer.
Another way to count last position is to slit string to array by delimeter equals to needed character and then substract length of characters
for the last element from the length of whole string
CREATE FUNCTION last_pos(char, text) RETURNS INTEGER AS
$$
select length($2) - length(a.arr[array_length(a.arr,1)])
from (select string_to_array($2, $1) as arr) as a
$$ LANGUAGE SQL;
For the first position it is easier to use
select position('#' in '2010-####-3434');