PostgreSql pass array argument in WHERE NOT - postgresql

I have a sql function that creates a materialized view:
CREATE FUNCTION reports_mt_views(exclude_ids int[]) RETURNS void AS
$BODY$
BEGIN
EXECUTE 'CREATE MATERIALIZED VIEW tx_materialized AS
SELECT tx.transaccion_id AS tx_transaccion_id,
...
WHERE creation_date > (current_date - interval '' 3 month '')
AND (account_id <> ALL (' || $1 || '))'
RETURN;
END;
$BODY$ LANGUAGE plpgsql STRICT;
I also tried as:
AND account_id NOT IN ' || $1|| ')'
But it does not work. When I execute:
SELECT reports_mt_views(ARRAY [1,2,8,538524]);
I have this error:
ERROR: operator does not exist: text || integer[]
LINE 171: AND (account_id <> ALL (' || $1 || '))'
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
There is not a problem with the array itself I tested it with a FOR loop and works. But into the condition it does not. What I have missed here?

The solution was to concanate the array as a text then it can be readby the clause where not in.
CREATE OR REPLACE FUNCTION pps.reports_mt_views(exclude_ids integer[])
...
DECLARE
in_accounts ALIAS FOR $1;
out_accounts TEXT;
BEGIN
out_accounts := in_accounts;
out_accounts := trim(leading '{' FROM out_accounts);
out_accounts := trim(trailing '}' FROM out_accounts);
And it can be used as:
WHERE (orden.cuenta_id NOT IN (' || out_accounts || '))

Related

Using array in dynamic commands inside PL/pgSQL function

In a PL/pgSQL function, I am creating a view using the EXECUTE statement. The where clause in the view takes as input some jenkins job names. These job names are passed to the function as a comma-separated string. They are then converted to an array so that they can be used as argument to ANY in the where clause. See basic code below:
CREATE OR REPLACE FUNCTION FETCH_ALL_TIME_AGGR_KPIS(jobs VARCHAR)
RETURNS SETOF GenericKPI AS $$
DECLARE
job_names TEXT[];
BEGIN
job_names = string_to_array(jobs,',');
EXECUTE 'CREATE OR REPLACE TEMP VIEW dynamicView AS ' ||
'with pipeline_aggregated_kpis AS (
select
jenkins_build_parent_id,
sum (duration) as duration
from test_all_finished_job_builds_enhanced_view where job_name = ANY (' || array(select quote_ident(unnest(job_names))) || ') and jenkins_build_parent_id is not null
group by jenkins_build_parent_id)
select ' || quote_ident('pipeline-job') || ' as job_name, b1.jenkins_build_id, pipeline_aggregated_kpis.status, pipeline_aggregated_kpis.duration FROM job_builds_enhanced_view b1 INNER JOIN pipeline_aggregated_kpis ON (pipeline_aggregated_kpis.jenkins_build_parent_id = b1.jenkins_build_id)';
RETURN QUERY (select
count(*) as total_executions,
round(avg (duration) FILTER (WHERE status = 'SUCCESS')::numeric,2) as average_duration
from dynamicView);
END
$$
LANGUAGE plpgsql;
The creation of the function is successful but an error message is returned when I try to call the function. See below:
eea_ci_db=> select * from FETCH_ALL_TIME_AGGR_KPIS('integration,test');
ERROR: malformed array literal: ") and jenkins_build_parent_id is not null
group by jenkins_build_parent_id)
select "
LINE 7: ...| array(select quote_ident(unnest(job_names))) || ') and jen...
^
DETAIL: Array value must start with "{" or dimension information.
CONTEXT: PL/pgSQL function fetch_all_time_aggr_kpis(character varying) line 8 at EXECUTE
It seems like there is something going wrong with quotes & the passing of an array of string. I tried all following options with the same result:
where job_name = ANY (' || array(select quote_ident(unnest(job_names))) || ') and jenkins_build_parent_id is not null
or
where job_name = ANY (' || quote_ident(job_names)) || ') and jenkins_build_parent_id is not null
or
where job_name = ANY (' || job_names || ') and jenkins_build_parent_id is not null
Any ideas?
Thank you
There is no need for dynamic SQL at all. There isn't even the need for PL/pgSQL to do this:
CREATE OR REPLACE FUNCTION FETCH_ALL_TIME_AGGR_KPIS(jobs VARCHAR)
RETURNS SETOF GenericKPI
AS
$$
with pipeline_aggregated_kpis AS (
select jenkins_build_parent_id,
sum (duration) as duration
from test_all_finished_job_builds_enhanced_view
where job_name = ANY (string_to_array(jobs,','))
and jenkins_build_parent_id is not null
group by jenkins_build_parent_id
), dynamic_view as (
select "pipeline-job" as job_name,
b1.jenkins_build_id,
pipeline_aggregated_kpis.status,
pipeline_aggregated_kpis.duration
FROM job_builds_enhanced_view b1
JOIN pipeline_aggregated_kpis
ON pipeline_aggregated_kpis.jenkins_build_parent_id = b1.jenkins_build_id
)
select count(*) as total_executions,
round(avg (duration) FILTER (WHERE status = 'SUCCESS')::numeric,2) as average_duration
from dynamic_view;
$$
language sql;
You could do this with PL/pgSQL as well, you just need to use RETURN QUERY WITH ....

Passing parameters gives array error in Postgres 8.2 function

I've got this function
/*
#Function: valiDates [Avoid inserting data from invalid dates]
#Purpose: Providing a _TABLE and a date _COLUMN to be validated.
Given a _DATE (from script name) validate that a certain % of data (_TOLERANCE) belongs to the _INTERVAL of days specified.
*/
CREATE OR REPLACE FUNCTION valiDates(_date date, _table regclass, _column text, _interval integer, _tolerance real) RETURNS BOOLEAN AS
$$
DECLARE result boolean;
BEGIN
EXECUTE 'SELECT
(SELECT count(*) FROM ' || _table::regclass || '
WHERE ' || _column || ' BETWEEN '''|| _date || '''::date and ''' || _date || '''::date + INTERVAL ''' || _interval || 'days'')'
|| '/
(SELECT COUNT(*) FROM ' || _table::regclass || ')::real
> ' || _tolerance
INTO result;
RETURN result;
END
;
$$ LANGUAGE plpgsql;
It actually works in my PostgreSQL environment Version 9.1.13, but when I try to call this function on Dev Server (PostgreSQL 8.2) the following error appears:
array value must start with "{" or dimension information
It should work on 8.2 as described in the official documentation page.
This is how I'm calling the function:
select valiDates('2015-03-01','_table','_date',1,0.8);
I really don't know how to fix it, I've already tried calling the function with '_table'::regclass but it doesn't works either.
Your error message most probably comes from _date as parameter name. _date is an alias for date[] - the type name for an array of dates. Similar for _interval.
Very old versions of Postgres could misunderstand that first word _date as data type instead of a parameter name. Use a sane parameter name to fix this. It is never a good idea to use basic type names as identifier of any kind.
Your function audited, should work in Postgres 8.2 (untested):
CREATE OR REPLACE FUNCTION validates(_day date, _table regclass, _column text
, _days integer, _tolerance real)
RETURNS BOOLEAN AS
$func$
DECLARE result boolean;
BEGIN
EXECUTE 'SELECT
(SELECT count(*) FROM ' || _table || '
WHERE ' || quote_ident(_column) || ' BETWEEN '''
|| _day || '''::date and '''
|| _day || '''::date + ' || _days
|| ') >
(SELECT COUNT(*) FROM ' || _table || ') * ' || _tolerance
INTO result;
RETURN result;
END
$func$ LANGUAGE plpgsql;
Notes:
Fix parameter names as discussed.
That's nonsense: _table::regclass - _table already is of type regclass
On the other hand, this is suicidal: _column. Wide open for SQL injection. Use quote_ident(_column) instead.
You can just add date + integer to add days. Much simpler.
3 / 2 > 1.5 can be rewritten to 3 > 2 * 1.5. Shorter, cleaner, clearer, avoids rounding error and no need for cast.
Modern syntax
In Postgres 9.1+, this could be:
CREATE OR REPLACE FUNCTION validates(_day date, _tbl regclass, _col text
, _days int, _tolerance real
, OUT result boolean) AS
$func$
BEGIN
EXECUTE format(
'SELECT count((%I BETWEEN $1 AND $2) OR NULL) > count(*) * $3 FROM %s'
, _col, _tbl
)
USING _day, _day + _days, _tolerance
INTO result;
END
$func$ LANGUAGE plpgsql;
Or even cleaner in Postgres 9.4+ with the new aggregate FILTER clause:
'SELECT count(*) FILTER (WHERE %I BETWEEN $1 AND $2)
> count(*) * $3 FROM %s'
Explanation:
How can I simplify this game statistics query?
Thank you Erwin for your advice, I took most of it. Finally I realized that it was expecting an array because of the _table parameter (regclass type) I just needed to change it or a string (text).

Strip parentheses from SQL query

I am tinkering with this snippet of SQL I reappropriated from another SO question:
CREATE OR REPLACE FUNCTION db_to_csv(path TEXT) RETURNS void AS $$
DECLARE
target RECORD;
statement TEXT;
BEGIN
FOR target IN
SELECT DISTINCT table_name
FROM information_schema.columns
WHERE table_schema='public'
AND position('_' in table_name) <> 1
ORDER BY 1
LOOP
statement := 'COPY '
|| target
|| ' TO '''
|| path
|| '/'
|| target
|| '.csv'
||''' DELIMITER '';'' CSV HEADER';
EXECUTE statement;
END LOOP;
return;
end;
$$ LANGUAGE plpgsql;
It very nearly almost works, but the statement it tries to execute looks like:
COPY (Bok_F_Acc) TO '/Users/jgt/dir/(Bok_F_Acc).csv' DELIMIT...
…Whereas I would like the statement to look like:
COPY Bok_F_Acc TO '/Users/jgt/dir/Bok_F_Acc.csv' DELIMIT...
How can I strip those parentheses?
SELECT QUOTE_IDENT(TRIM(BOTH '()' FROM target));
Note that this would remove any number of leading/trailing parentheses.
To remove at most one of each, you could use :
SELECT QUOTE_IDENT(REGEXP_REPLACE(target, E'\^\\(|\\)\$', '', 'g'));

PostgreSQL: ERROR operator does not exist: || character varying

I need to check the condition within function using string_agg() function and need to assign it to variable. After assigning I need to execute the variable with value.
Example:
create or replace function funct1(a int,b varchar)
returns void as
$$
declare
wrclause varchar := '';
sqlq varchar ;
t varchar;
begin
IF (b IS NOT NULL ) THEN
wrclause := 'AND b IN ('|| b || ')';
END IF;
sqlq := string_agg('select *, abcd as "D" from ' ||table_namess,' Union all ') as namess
from tablescollection2 ud
inner join INFORMATION_SCHEMA.Tables so on ud.table_namess = so.Table_name
WHERE cola NOT IN (SELECT cola FROM tablet WHERE colb = || a ||) || wrclause; /* Error occurred here at = || a */
raise info '%',sqlq;
execute sqlq into t;
raise info '%',t;
end;
$$
language plpgsql;
Calling Function:
select funct1(1,'1,2,3');
Error:
ERROR: operator does not exist: || integer
|| is an operator for catenating two pieces of text, it requires you to have text (or something convertible to text) both before and after the operator, like so:
select 'a' || 'b'
select 'a' || 3
So while these seem to be valid:
wrclause := 'AND b IN ('|| b || ')';
sqlq := string_agg('select *, abcd as "D" from ' ||table_namess,' Union all ') as namess
This is definitely not:
WHERE cola NOT IN (SELECT cola FROM tablet WHERE colb = || a ||) || wrclause;
What were you trying to achieve here?
It looks like you may be trying to construct a query dynamically. You need to remember that you cannot mix free text with SQL and expect Postgres to sort it out, no programming or query language does that.
If that's your intention, you should construct the query string first in its entirety (in a variable), and then call EXECUTE with it to have it interpreted.
Have a look at these:
Postgres Dynamic Query Function
PostgreSQL - dynamic value as table name
This piece contains the syntax error
... IN (SELECT cola FROM tablet WHERE colb = || a ||) || ...
PostgreSQL can understand this, but will try to search for unary prefix (and a postfix) || operator, which are not exist by default (they can be created however, but the error message says, that's not the case)
Edit:
F.ex. these are valid (predefined) unary operators on numbers:
SELECT |/ 25.0, -- prefix, square root, result: 5.0
5 !, -- postfix, factorial, result: 120,
# -5, -- prefix, absolute, result: 5
# -5 !; -- mixed, result: 120

PostgreSQL: Syntax error at or near "||"

I am creating function which return the select query result from it. The details as shown below in the example:
Example:
Create or replace function fun_test(cola text,colb text,rel text)
returns table(columna text,columnb text)as
$Body$
Declare
table_name varchar :='Table_';
Begin
table_name := table_name || rel;
return query select distinct || quote_ident(cola) ||,||quote_ident(colb)|| from || quote_ident(table_name) ;
end;
$Body$
language plpgsql;
Error:
ERROR: syntax error at or near "||"
LINE 10: ...| quote_ident(cola) ||,||quote_ident(colb)|| from || quote_i...
^
You're trying to construct a query dynamically, but you're not using EXECUTE. That won't work. You can't just put arbitrary expressions in place of identifiers, like:
return query select distinct || quote_ident(cola) ||,||quote_ident(colb)|| from || quote_ident(table_name) ;
which is why you're getting the error at:
from ||
^
as that's syntactically invalid nonsense.
Instead I think you want:
RETURN QUERY EXECUTE 'select distinct ' || quote_ident(cola) ||', '||quote_ident(colb)||' from '|| quote_ident(table_name);
which is better written as:
RETURN QUERY EXECUTE format('select distinct %I, %I from %I', cola, colb, table_name);