Using IN vs ANY in CREATE FUNCTION - postgresql

PostgreSQL 9.5
I'm trying to wrap my head around how I can go create a function that will work similar to this but unable to make any headway. What do I need to tweak to make this work appropriately.
SELECT * FROM get_all_rows ('{Home,Away}','{LF,RL,CENTER}','2016-06-01 00:00:00','2016-06-30 23:59:59')
-- '2016-06-08 12:24:50','2016-06-09 04:59:45','SW'
-- '2016-06-08 07:12:12','2016-06-09 08:55:25','NW'
CREATE OR REPLACE FUNCTION get_all_rows (
tbl TEXT,
disp TEXT[],
area TEXT[],
current TIMESTAMP,
future TIMESTAMP
)
RETURNS TABLE (
a TIMESTAMP,
b TIMESTAMP,
c TEXT
) AS
$func$
BEGIN
EXECUTE
'SELECT
door_time,
guard_time,
area
FROM
' || quote_ident(tbl) || '
WHERE
disposition = ANY (disp)
and area = ANY (area)
and door_time IS NOT NULL
and guard_time IS NOT NULL
and arrival >= arrival_begin
and arrival <= arrival_end';
END
$func$ LANGUAGE plpgsql;

There are several errors in what you did:
You didn't supply the first argument tbl to your function call.
You used current and futureas parameters, but in the function body they are called arrival_begin and arrival_end (at least that's what I assume).
You didn't add the values of disp, area, current and future to the query, but the names of the variables.
A fixed version of the function definition might look like this:
CREATE OR REPLACE FUNCTION get_all_rows (
tbl TEXT,
disp TEXT,
area TEXT,
current TIMESTAMP,
future TIMESTAMP
)
RETURNS TABLE (
a TIMESTAMP,
b TIMESTAMP,
c TEXT) AS
$func$
BEGIN
RETURN QUERY EXECUTE
'SELECT
door_time,
guard_time,
area
FROM
' || quote_ident(tbl) || '
WHERE
disposition = ANY (' || quote_literal(disp) || ')
and area = ANY (' || quote_literal(area) || ')
and door_time IS NOT NULL
and guard_time IS NOT NULL
and arrival >= ' || quote_literal(current) || '
and arrival <= ' || quote_literal(future);
END
$func$ LANGUAGE plpgsql;

Related

Syntac Error when using EXECUTE command in postgresql Function

I'm trying to write a function which executes a dynamically prepared sql query and return the result as table.
I was refering to the SO answer, it mentioned to use language plpgsql, even after using it I'm getting the same syntax error.
Below is the function code provided.
CREATE OR REPLACE FUNCTION public.execute_test(
ddsmappingids text,
totalnumberofrecords bigint,
skiprecords bigint,
pagesize integer,
cid bigint)
RETURNS TABLE(sampletime timestamp without time zone, jsonstring text, rowscount bigint)
LANGUAGE 'plpgsql'
COST 100
VOLATILE PARALLEL UNSAFE
ROWS 1000
AS $BODY$
DECLARE
table_names text:='dynamictable';
sql_query text;
dsids_array int[];
sub_query text;
BEGIN
if cid>=2 then
select 'dynamictable_'||cid into table_names;
end if;
select string_to_array(ddsmappingids, ',')::int[] into dsids_array;
sub_query:='SELECT * , COUNT(*) over() AS row_count FROM (SELECT start_time, cast(jsonb_object_agg(vw.dataseries_name, ROUND(CAST(o.dbl_value as numeric), 4)) as text) AS DataSeriesValue
FROM ' || quote_ident(table_names) ||' o
join vwdds vw on o.ddmid=vw.id    
where ddmid in (' || array_to_string(dsids_array,',') || ')
and o.dbl_value is not null
GROUP BY start_time
ORDER BY start_time desc
limit ' || totalnumberofrecords || ') t offset '|| skiprecords || ' rows fetch next ' || pagesize || ' rows only;';
RAISE NOTICE 'Temporary table created';
RETURN QUERY Execute sub_query;
END
$BODY$;
Error Message when running the function
EXECUTE command, when i try with some simple query it is working fine, but the query mentioned in the code snippet it is giving syntax error.
Please help me where I'm doing wrong.

EXTRACT INTO with multiple rows (PostgreSQL)

this is my function:
CREATE OR REPLACE FUNCTION SANDBOX.DAILYVERIFY_DATE(TABLE_NAME regclass, DATE_DIFF INTEGER)
RETURNS void AS $$
DECLARE
RESULT BOOLEAN;
DATE DATE;
BEGIN
EXECUTE 'SELECT VORHANDENES_DATUM AS DATE, CASE WHEN DATUM IS NULL THEN FALSE ELSE TRUE END AS UPDATED FROM
(SELECT DISTINCT DATE VORHANDENES_DATUM FROM ' || TABLE_NAME ||
' WHERE DATE > CURRENT_DATE -14-'||DATE_DIFF|| '
) A
RIGHT JOIN
(
WITH compras AS (
SELECT ( NOW() + (s::TEXT || '' day'')::INTERVAL )::TIMESTAMP(0) AS DATUM
FROM generate_series(-14, -1, 1) AS s
)
SELECT DATUM::DATE
FROM compras)
B
ON DATUM = VORHANDENES_DATUM'
INTO date,result;
RAISE NOTICE '%', result;
INSERT INTO SANDBOX.UPDATED_TODAY VALUES (TABLE_NAME, DATE, RESULT);
END;
$$ LANGUAGE plpgsql;
It is supposed to upload rows into the table SANDBOX.UPDATED_TODAY, which contains table name, a date and a boolean.
The boolean shows, whether there was an entry for that date in the table. The whole part, which is inside of EXECUTE ... INTO works fine and gives me those days.
However, this code only inserts the first row of the query's result. What I want is that all 14 rows get inserted. Obviously, I need to change it into something like a loop or something completely different, but how exactly would that work?
Side note: I removed some unnecessary parts regarding those 2 parameters you can see. It does not have to do with that at all.
Put the INSERT statement inside the EXECUTE. You don't need the result of the SELECT for anything other than inserting it into that table, right? So just insert it directly as part of the same query:
CREATE OR REPLACE FUNCTION SANDBOX.DAILYVERIFY_DATE(TABLE_NAME regclass, DATE_DIFF INTEGER)
RETURNS void AS
$$
BEGIN
EXECUTE
'INSERT INTO SANDBOX.UPDATED_TODAY
SELECT ' || QUOTE_LITERAL(TABLE_NAME) || ', VORHANDENES_DATUM, CASE WHEN DATUM IS NULL THEN FALSE ELSE TRUE END
FROM (
SELECT DISTINCT DATE VORHANDENES_DATUM FROM ' || TABLE_NAME ||
' WHERE DATE > CURRENT_DATE -14-'||DATE_DIFF|| '
) A
RIGHT JOIN (
WITH compras AS (
SELECT ( NOW() + (s::TEXT || '' day'')::INTERVAL )::TIMESTAMP(0) AS DATUM
FROM generate_series(-14, -1, 1) AS s
)
SELECT DATUM::DATE
FROM compras
) B
ON DATUM = VORHANDENES_DATUM';
END;
$$ LANGUAGE plpgsql;
The idiomatic way to loop through dynamic query results would be
FOR date, result IN
EXECUTE 'SELECT ...'
LOOP
INSERT INTO ...
END LOOP;

Using array in dynamic commands inside PL/pgSQL function

In a PL/pgSQL function, I am creating a view using the EXECUTE statement. The where clause in the view takes as input some jenkins job names. These job names are passed to the function as a comma-separated string. They are then converted to an array so that they can be used as argument to ANY in the where clause. See basic code below:
CREATE OR REPLACE FUNCTION FETCH_ALL_TIME_AGGR_KPIS(jobs VARCHAR)
RETURNS SETOF GenericKPI AS $$
DECLARE
job_names TEXT[];
BEGIN
job_names = string_to_array(jobs,',');
EXECUTE 'CREATE OR REPLACE TEMP VIEW dynamicView AS ' ||
'with pipeline_aggregated_kpis AS (
select
jenkins_build_parent_id,
sum (duration) as duration
from test_all_finished_job_builds_enhanced_view where job_name = ANY (' || array(select quote_ident(unnest(job_names))) || ') and jenkins_build_parent_id is not null
group by jenkins_build_parent_id)
select ' || quote_ident('pipeline-job') || ' as job_name, b1.jenkins_build_id, pipeline_aggregated_kpis.status, pipeline_aggregated_kpis.duration FROM job_builds_enhanced_view b1 INNER JOIN pipeline_aggregated_kpis ON (pipeline_aggregated_kpis.jenkins_build_parent_id = b1.jenkins_build_id)';
RETURN QUERY (select
count(*) as total_executions,
round(avg (duration) FILTER (WHERE status = 'SUCCESS')::numeric,2) as average_duration
from dynamicView);
END
$$
LANGUAGE plpgsql;
The creation of the function is successful but an error message is returned when I try to call the function. See below:
eea_ci_db=> select * from FETCH_ALL_TIME_AGGR_KPIS('integration,test');
ERROR: malformed array literal: ") and jenkins_build_parent_id is not null
group by jenkins_build_parent_id)
select "
LINE 7: ...| array(select quote_ident(unnest(job_names))) || ') and jen...
^
DETAIL: Array value must start with "{" or dimension information.
CONTEXT: PL/pgSQL function fetch_all_time_aggr_kpis(character varying) line 8 at EXECUTE
It seems like there is something going wrong with quotes & the passing of an array of string. I tried all following options with the same result:
where job_name = ANY (' || array(select quote_ident(unnest(job_names))) || ') and jenkins_build_parent_id is not null
or
where job_name = ANY (' || quote_ident(job_names)) || ') and jenkins_build_parent_id is not null
or
where job_name = ANY (' || job_names || ') and jenkins_build_parent_id is not null
Any ideas?
Thank you
There is no need for dynamic SQL at all. There isn't even the need for PL/pgSQL to do this:
CREATE OR REPLACE FUNCTION FETCH_ALL_TIME_AGGR_KPIS(jobs VARCHAR)
RETURNS SETOF GenericKPI
AS
$$
with pipeline_aggregated_kpis AS (
select jenkins_build_parent_id,
sum (duration) as duration
from test_all_finished_job_builds_enhanced_view
where job_name = ANY (string_to_array(jobs,','))
and jenkins_build_parent_id is not null
group by jenkins_build_parent_id
), dynamic_view as (
select "pipeline-job" as job_name,
b1.jenkins_build_id,
pipeline_aggregated_kpis.status,
pipeline_aggregated_kpis.duration
FROM job_builds_enhanced_view b1
JOIN pipeline_aggregated_kpis
ON pipeline_aggregated_kpis.jenkins_build_parent_id = b1.jenkins_build_id
)
select count(*) as total_executions,
round(avg (duration) FILTER (WHERE status = 'SUCCESS')::numeric,2) as average_duration
from dynamic_view;
$$
language sql;
You could do this with PL/pgSQL as well, you just need to use RETURN QUERY WITH ....

Passing parameters gives array error in Postgres 8.2 function

I've got this function
/*
#Function: valiDates [Avoid inserting data from invalid dates]
#Purpose: Providing a _TABLE and a date _COLUMN to be validated.
Given a _DATE (from script name) validate that a certain % of data (_TOLERANCE) belongs to the _INTERVAL of days specified.
*/
CREATE OR REPLACE FUNCTION valiDates(_date date, _table regclass, _column text, _interval integer, _tolerance real) RETURNS BOOLEAN AS
$$
DECLARE result boolean;
BEGIN
EXECUTE 'SELECT
(SELECT count(*) FROM ' || _table::regclass || '
WHERE ' || _column || ' BETWEEN '''|| _date || '''::date and ''' || _date || '''::date + INTERVAL ''' || _interval || 'days'')'
|| '/
(SELECT COUNT(*) FROM ' || _table::regclass || ')::real
> ' || _tolerance
INTO result;
RETURN result;
END
;
$$ LANGUAGE plpgsql;
It actually works in my PostgreSQL environment Version 9.1.13, but when I try to call this function on Dev Server (PostgreSQL 8.2) the following error appears:
array value must start with "{" or dimension information
It should work on 8.2 as described in the official documentation page.
This is how I'm calling the function:
select valiDates('2015-03-01','_table','_date',1,0.8);
I really don't know how to fix it, I've already tried calling the function with '_table'::regclass but it doesn't works either.
Your error message most probably comes from _date as parameter name. _date is an alias for date[] - the type name for an array of dates. Similar for _interval.
Very old versions of Postgres could misunderstand that first word _date as data type instead of a parameter name. Use a sane parameter name to fix this. It is never a good idea to use basic type names as identifier of any kind.
Your function audited, should work in Postgres 8.2 (untested):
CREATE OR REPLACE FUNCTION validates(_day date, _table regclass, _column text
, _days integer, _tolerance real)
RETURNS BOOLEAN AS
$func$
DECLARE result boolean;
BEGIN
EXECUTE 'SELECT
(SELECT count(*) FROM ' || _table || '
WHERE ' || quote_ident(_column) || ' BETWEEN '''
|| _day || '''::date and '''
|| _day || '''::date + ' || _days
|| ') >
(SELECT COUNT(*) FROM ' || _table || ') * ' || _tolerance
INTO result;
RETURN result;
END
$func$ LANGUAGE plpgsql;
Notes:
Fix parameter names as discussed.
That's nonsense: _table::regclass - _table already is of type regclass
On the other hand, this is suicidal: _column. Wide open for SQL injection. Use quote_ident(_column) instead.
You can just add date + integer to add days. Much simpler.
3 / 2 > 1.5 can be rewritten to 3 > 2 * 1.5. Shorter, cleaner, clearer, avoids rounding error and no need for cast.
Modern syntax
In Postgres 9.1+, this could be:
CREATE OR REPLACE FUNCTION validates(_day date, _tbl regclass, _col text
, _days int, _tolerance real
, OUT result boolean) AS
$func$
BEGIN
EXECUTE format(
'SELECT count((%I BETWEEN $1 AND $2) OR NULL) > count(*) * $3 FROM %s'
, _col, _tbl
)
USING _day, _day + _days, _tolerance
INTO result;
END
$func$ LANGUAGE plpgsql;
Or even cleaner in Postgres 9.4+ with the new aggregate FILTER clause:
'SELECT count(*) FILTER (WHERE %I BETWEEN $1 AND $2)
> count(*) * $3 FROM %s'
Explanation:
How can I simplify this game statistics query?
Thank you Erwin for your advice, I took most of it. Finally I realized that it was expecting an array because of the _table parameter (regclass type) I just needed to change it or a string (text).

Structure of query does not match function result type, RETURNS TABLE

I need a simple function to return dynamic set of columns. I've found couple of examples on SO and end up with the following:
CREATE or replace FUNCTION getColumn(_column1 text, _column2 text, _column3 text, _table text)
RETURNS TABLE(cmf1 text, cmf2 text, cmf3 text) AS $$
BEGIN
RETURN QUERY EXECUTE
'SELECT '
|| quote_ident(_column1)::text || ' as cmf1,'
|| quote_ident(_column2)::text || ' as cmf2,'
|| quote_ident(_column3)::text || ' as cmf3'
' FROM '
|| quote_ident(_table);
END;
$$ LANGUAGE plpgsql;
I need this function to work only with varchar/text columns so I created this testing table:
create table test20130205 (
a text,
b text,
c varchar,
d text)
;
Finally, I can run some tests:
select * from getColumn('a','b','d','test20130205');
-- ok
select * from getColumn('a','b','c','test20130205');
-- error
ERROR: structure of query does not match function result type
DETAIL: Returned type character varying does not match expected type text in column 3.
CONTEXT: PL/pgSQL function getcolumn(text,text,text,text) line 3 at RETURN QUERY
It seems like type for column c (varchar) is checked before cast - this seems strange, but I guess I've missed something.
How can I fix my function?
(PostgreSQL 9.1)
In your current function, the casts to text do not apply to the output columns values, they apply to their names (the result of quote_ident).
The cast should be moved inside the query itself:
CREATE or replace FUNCTION getColumn(_column1 text, _column2 text, _column3 text, _table text)
RETURNS TABLE(cmf1 text, cmf2 text, cmf3 text) AS $$
BEGIN
RETURN QUERY EXECUTE
'SELECT '
|| quote_ident(_column1) || '::text as cmf1,'
|| quote_ident(_column2) || '::text as cmf2,'
|| quote_ident(_column3) || '::text as cmf3'
' FROM '
|| quote_ident(_table);
END;
$$ LANGUAGE plpgsql;