How to use variable inside quoted string in postgres dynamic SQL - postgresql

I created the following function with plpgsql which takes 3 parameters.
CREATE OR REPLACE FUNCTION public.most_service_calls(
comp_id integer,
calls integer,
months integer)
RETURNS TABLE(state character varying, city character varying, cust_name character varying, num_calls bigint, cost numeric)
LANGUAGE 'plpgsql'
COST 100
VOLATILE
ROWS 1000
AS $BODY$
Begin
return query execute
'select * from
(select l.state, l.city, l.cust_name, count(distinct a.svc_ord_nbr) num_calls,
round(avg(a.std_labr_net_amt_dcrncy) + avg(a.travel_net_amt_dcrncy), 2)
from dmt_mas_svc_ord_fact a
inner join dmt_mas_cust_dim b on a.shipto_cust_id = b.cust_id
inner join store_location l on b.cust_name = l.cust_name
inner join company co on b.cust_lvl_2_nbr = co.company_nbr
where b.sap_sls_org_name like ''%Stanley US%''
and a.create_dtm >= now() - interval '' $3 months''
and co.company_id = $1
group by l.state, l.city, l.cust_name
order by l.state, l.city, l.cust_name ) q
where num_calls >= $2'
using comp_id, calls, months;
end;
$BODY$;
Since the query is quoted, all the single quoted strings are double quoted. Three variables are represented by $1, $2, $3. It is the variable inside a string that is causing the trouble. a.create_dtm >= now() - interval '' $3 months''
When I run the function, it seems to ignore whatever third parameter I provided. Therefore, all the following return the same result.
select * from most_service_calls(1,5,1)
select * from most_service_calls(1,5,12)
select * from most_service_calls(1,5,24)
And it turned out, $3 inside '' '' is taken as 3 since the result matches that of the query with 3 months hardcoded in the query.
What is the correct way to include the variable inside a string in a quoted query like this?

Your problem is not specific to dynamic SQL - you can't refer to a placeholder within a quoted string even in a normal SQL query.
Instead you could use:
$3 * interval '1 month'
or:
($3 || ' months')::interval
The first form multiplies your supplied numeric value by a one month interval. The second constructs a string specifying the number of months and then casts it to an interval.

Related

postgres function returns operator doesn't exist integer - timestamp

Hi I have a function that when calling it to test returns the error in the question. i know it is to do with how i insert the variable into part of the query but i cannot work out why, i've tried changing the type castings around on it but no luck. any help would be appreciated
create or replace function ageing_balance_calc(start_date text, client_id text) returns void
AS $BODY$
declare
_absd date;
_client_id text;
req text;
BEGIN
_absd:=(to_date(start_date,'YYYY-MM-DD'));
_client_id:= client_id;
with cte as
(
Select * from crosstab('
select * from (
select absd,
case when a.days_ago between 0 and 30 then ''0-30''
when a.days_ago between 31 and 60 then ''31-60''
when a.days_ago between 61 and 90 then ''61-90''
when a.days_ago >90 then ''90+''
else ''not due'' end as bucket, sum(a.item_amount)
from (select date_part(''day'',((due_date::timestamp) - ('||_absd||' ::text::timestamp)) )as days_ago, item_amount, '||_absd||'::text::date as absd from fdw_test) a
group by bucket, absd)b order by 1,2') as (absd date,
bucket0_30 numeric,
bucket31_60 numeric,
bucket61_90 numeric,
bucket90_plus numeric,
not_due numeric)
)
update ddva.kpi_calc_results
set
absd = a.absd,
bucket0_30 = a.bucket0_30,
bucket31_60 = a.bucket31_60,
bucket61_90 = a.bucket61_90,
bucket90_plus = a.bucket90_plus,
not_due = a.not_due from (
select absd, bucket0_30, bucket31_60, bucket61_90, bucket90_plus, not_due from cte) as a
where ddva.kpi_calc_results.client_id = _client_id ;
END;
$BODY$
LANGUAGE plpgsql
update
the date_part works if i remove it to its own query and then use $date to insert the date as a string that way. if i remove the to_date part at the start of the function i still get the same error as before. does '||x||' do anything to the data type?
the problem is here:
the result query looks like this: (2021-07-07::text::timestamp) and results in error: iteger - timestamp.
You must add more quotes: ('''||_absd||'''::text::timestamp)
Well:
date_part(''day'',((due_date::timestamp)
is going to return an integer:
select date_part('day', now());
27
I'm guessing you want something more like:
select now()::date; 2021-07-27
UPDATE
This part:
'''||_absd||'''::text::timestamp
won't work either:
select '''||_absd||'''::text::timestamp;
ERROR: invalid input syntax for type timestamp: "'||_absd||'"
the below line:
(select date_part(''day'',((due_date::timestamp) - ('||_absd||' ::text::timestamp)) )as days_ago, item_amount, '||_absd||'::text::date as absd from fdw_test)
was changed slightly to this:
(select date_part(''day'',((due_date::timestamp) - ('''||_absd||'''::timestamp)) )as days_ago, item_amount, '''||_absd||'''::date as absd from fdw_test)
the function now works as expected

Dynamically add a column with multiple values to any table using a PL/pgSQL function

I would like to use a function/procedure to add to a 'template' table an additional column (e.g. period name) with multiple values, and do a cartesian product on the rows, so my 'template' is duplicated with the different values provided for the new column.
E.g. Add a period column with 2 values to my template_country_channel table:
SELECT *
FROM unnest(ARRAY['P1', 'P2']) AS prd(period)
, template_country_channel
ORDER BY period DESC
, sort_cnty
, sort_chan;
/*
-- this is equivalent to:
(
SELECT 'P2'::text AS period
, *
FROM template_country_channel
) UNION ALL (
SELECT 'P1'::text AS period
, *
FROM template_country_channel
)
--
*/
This query is working fine, but I was wondering if I could turn that into a PL/pgSQL function/procedure, providing the new column values to be added, the column to add the extra column to (and optionally specify the order by conditions).
What I would like to do is:
SELECT *
FROM template_with_periods(
'template_country_channel' -- table name
, ARRAY['P1', 'P2'] -- values for the new column to be added
, 'period DESC, sort_cnty, sort_chan' -- ORDER BY string (optional)
);
and have the same result as the 1st query.
So I created a function like:
CREATE OR REPLACE FUNCTION template_with_periods(template regclass, periods text[], order_by text)
RETURNS SETOF RECORD
AS $BODY$
BEGIN
RETURN QUERY EXECUTE 'SELECT * FROM unnest($2) AS prd(period), $1 ORDER BY $3' USING template, periods, order_by ;
END;
$BODY$
LANGUAGE 'plpgsql'
;
But when I run:
SELECT *
FROM template_with_periods('template_country_channel', ARRAY['P1', 'P2'], 'period DESC, sort_cnty, sort_chan');
I have the error ERROR: 42601: a column definition list is required for functions returning “record”
After some googling, it seems that I need to define the list of columns and types to perform the RETURN QUERY (as the error message precisely states).
Unfortunately, the whole idea is to use the function with many 'template' tables, so columns name & type lists is not fixed.
Is there any other approach to try?
Or is the only way to make it work is to have within the function, a way to get list of columns' names and types of the template table?
I did this with refcursor if You want output columns list completely dynamic:
CREATE OR REPLACE FUNCTION is_record_exists(tablename character varying, columns character varying[], keepcolumns character varying[] DEFAULT NULL::character varying[])
RETURNS SETOF refcursor AS
$BODY$
DECLARE
ref refcursor;
keepColumnsList text;
columnsList text;
valuesList text;
existQuery text;
keepQuery text;
BEGIN
IF keepcolumns IS NOT NULL AND array_length(keepColumns, 1) > 0 THEN
keepColumnsList := array_to_string(keepColumns, ', ');
ELSE
keepColumnsList := 'COUNT(*)';
END IF;
columnsList := (SELECT array_to_string(array_agg(name || ' = ' || value), ' OR ') FROM
(SELECT unnest(columns[1:1]) AS name, unnest(columns[2:2]) AS value) pair);
existQuery := 'SELECT ' || keepColumnsList || ' FROM ' || tableName || ' WHERE ' || columnsList;
RAISE NOTICE 'Exist query: %', existQuery;
OPEN ref FOR EXECUTE
existQuery;
RETURN next ref;
END;$BODY$
LANGUAGE plpgsql;
Then need to call FETCH ALL IN to get results. Detailed syntax here or there: https://stackoverflow.com/a/12483222/630169. Seems it is the only way for now. Hope something will be changed in PostgreSQL 11 with PROCEDURES.

Month and year instead of the date variable

I work with PostgreSQL and I have three variables in a function and one of them is of type date:
CREATE OR REPLACE FUNCTION public.fn_reporte_venta_mes(
IN p_fech = date,
IN p_client integer,
IN p_comprobante character
The parameter p_fech serves to enter a date from PHP in format '1111-11-11'. Instead, I want to only insert date a month and year format '1111-11'. But I do not know how to change the data type of the p_fech variable since when I put only the day and month in PHP I get compatibility error.
My complete function:
CREATE OR REPLACE FUNCTION public.fn_reporte_venta(
IN p_fech date,
IN p_client integer,
IN p_comprobante character)
RETURNS TABLE(nro integer, fecha date, tipo character varying, cliente text, porc_igv numeric, st numeric, igv numeric, total numeric) AS
$BODY$
begin
return query
select v.numero_venta,
v.fecha_venta,
tc.descripcion,
concat(apellido_paterno, ' ', apellido_materno, ' ', nombres) as client,
v.porcentaje_igv,
v.sub_total,
v.igv,
v.total
from venta v
inner join cliente cl on v.codigo_cliente = cl.codigo_cliente
inner join tipo_comprobante tc on v.codigo_tipo_comprobante = tc.codigo_tipo_comprobante
where v.estado = 'E' and
(
case when p_fech = '11-11-1111' then
1 = 1
else
v.fecha_venta = p_fech
end
)
and
(
case when p_client = 0 then
1 = 1
else
cl.codigo_cliente = p_client
end
) and
(
case when p_comprobante = '00' then
1 = 1
else
tc.codigo_tipo_comprobante = p_comprobante
end
)
order by 2;
end
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100
ROWS 1000;
Use a character type parameter (text or varchar, not character!) and the function to_date() to convert the input to a date.
Your function, simplified and fixed:
CREATE OR REPLACE FUNCTION public.fn_reporte_venta(p_fech text
, p_client integer
, p_comprobante text)
RETURNS TABLE(nro integer, fecha date, tipo varchar, cliente text
, porc_igv numeric, st numeric, igv numeric, total numeric) AS
$func$
DECLARE
v_fech date := to_date(p_fech, 'YYYY-MM-DD'); -- transform to date
BEGIN
RETURN QUERY
SELECT v.numero_venta,
v.fecha_venta,
t.descripcion,
concat_ws(' ', c.apellido_paterno, c.apellido_materno, c.nombres) AS client,
v.porcentaje_igv,
v.sub_total,
v.igv,
v.total
FROM venta v
JOIN cliente c USING (codigo_cliente)
JOIN tipo_comprobante t USING (codigo_tipo_comprobante)
WHERE v.estado = 'E'
AND (v_fech = '1111-11-11' OR v.fecha_venta = v_fech) -- var goes here
AND (p_client = 0 OR c.codigo_cliente = p_client)
AND (p_comprobante = '00' OR t.codigo_tipo_comprobante = p_comprobante)
ORDER BY 2;
END
$func$ LANGUAGE plpgsql STABLE ROWS 1000;
Always use the ISO-8601 format for date literals (YYYY-MM-DD), which is unambiguous with any locale setting. Don't fall for local syntax dialects, they break if a session should run with a different locale.
I am using a date variable in the plpgsql function, which can be assigned at declaration right away.
This expression is very versatile:
to_date(p_fech, 'YYYY-MM-DD');
to_date() allows to omit elements to the right to 1 for missing data. All of these are valid and result in '2016-01-01':
SELECT to_date('2016-01-01', 'YYYY-MM-DD')
, to_date('2016-1-1' , 'YYYY-MM-DD')
, to_date('2016-01' , 'YYYY-MM-DD')
, to_date('2016-' , 'YYYY-MM-DD')
, to_date('2016' , 'YYYY-MM-DD');
So you can pass in full dates or truncated to month or even year. You always specify a single day this way.
Or to always cover a whole month:
...
v_fech date := to_date(p_fech, 'YYYY-MM'); -- no "-DD" truncates to month
...
AND (v_fech = '1111-11-01' -- also truncated to 1st of month!
OR v.fecha_venta >= v_fech
AND v.fecha_venta < v_fech + interval '1 month')
...
Why not data type character? Because that's an outdated, largely useless type, typically a misunderstanding:
Best way to check for "empty or null value"
Any downsides of using data type "text" for storing strings?
Also note how I replaced your verbose CASE statements with simple OR expressions.
And some other minor improvements ...

Why postgres function so slow but single query is fast?

I have function to get employee in 'Create' status.
CREATE OR REPLACE FUNCTION get_probation_contract(AccountOrEmpcode TEXT, FromDate DATE,
ToDate DATE)
RETURNS TABLE("EmpId" INTEGER, "EmpCode" CHARACTER VARYING,
"DomainAccount" CHARACTER VARYING, "JoinDate" DATE,
"ContractTypeCode" CHARACTER VARYING, "ContractTypeName" CHARACTER VARYING,
"ContractFrom" DATE, "ContractTo" DATE, "ContractType" CHARACTER VARYING,
"Signal" CHARACTER VARYING) AS $$
BEGIN
RETURN QUERY
EXECUTE 'SELECT
he.id "EmpId",
rr.code "EmpCode",
he.login "DomainAccount",
he.join_date "JoinDate",
contract_type.code "ContractTypeCode",
contract_type.name "ContractTypeName",
contract.date_start "ContractFrom",
contract.date_end "ContractTo",
CASE WHEN contract_group.code = ''1'' THEN ''Probation''
WHEN contract_group.code IN (''3'', ''4'', ''5'') THEN ''Official''
WHEN contract_group.code = ''2'' THEN ''Collaborator'' END :: CHARACTER VARYING "ContractType",
''CREATE'' :: CHARACTER VARYING "Signal"
FROM
hr_employee he
INNER JOIN resource_resource rr
ON rr.id = he.resource_id
INNER JOIN hr_contract contract
ON contract.employee_id = he.id AND contract.date_start = (
SELECT max(date_start) "date_start"
FROM hr_contract cc
WHERE cc.employee_id = contract.employee_id
)
INNER JOIN hr_contract_type contract_type
ON contract_type.id = contract.type_id
INNER JOIN hr_contract_type_group contract_group
ON contract_group.id = contract_type.contract_type_group_id
WHERE
contract_group.code = ''1''
AND
($1 IS NULL OR $1 = '''' OR rr.code = $1 OR
he.login = $1)
AND (
(he.join_date BETWEEN $2 AND $3)
OR (he.join_date IS NOT NULL AND (contract.date_start BETWEEN $2 AND $3))
OR (he.create_date BETWEEN $2 AND $3 AND he.create_date > he.join_date)
)
AND rr.active = TRUE
'using AccountOrEmpcode, FromDate, ToDate ;
END;
$$ LANGUAGE plpgsql;
It took 37 second to execute
SELECT *
FROM get_probation_contract('', '2014-01-01', '2014-06-01');
When I use single query
SELECT
he.id "EmpId",
rr.code "EmpCode",
he.login "DomainAccount",
he.join_date "JoinDate",
contract_type.code "ContractTypeCode",
contract_type.name "ContractTypeName",
contract.date_start "ContractFrom",
contract.date_end "ContractTo",
CASE WHEN contract_group.code = '1' THEN 'Probation'
WHEN contract_group.code IN ('3', '4', '5') THEN 'Official'
WHEN contract_group.code = '2' THEN 'Collaborator' END :: CHARACTER VARYING "ContractType",
'CREATE' :: CHARACTER VARYING "Signal"
FROM
hr_employee he
INNER JOIN resource_resource rr
ON rr.id = he.resource_id
INNER JOIN hr_contract contract
ON contract.employee_id = he.id AND contract.date_start = (
SELECT max(date_start) "date_start"
FROM hr_contract
WHERE employee_id = he.id
)
INNER JOIN hr_contract_type contract_type
ON contract_type.id = contract.type_id
INNER JOIN hr_contract_type_group contract_group
ON contract_group.id = contract_type.contract_type_group_id
WHERE
contract_group.code = '1'
AND (
(he.join_date BETWEEN '2014-01-01' AND '2014-06-01')
OR (he.join_date IS NOT NULL AND (contract.date_start BETWEEN '2014-01-01' AND '2014-01-06'))
OR (he.create_date BETWEEN '2014-01-01' AND '2014-01-06' AND he.create_date > he.join_date)
)
AND rr.active = TRUE
It take 5 second to complete
How to optimize the function above.
and why function is slow than single query so much even I use execute 'select ...' in function.
Indexing in field id each table.
Possible reason is a blind optimization for prepared statements (embedded SQL). It is little bit better in new PostgreSQL releases, although it can be the issue there too. Execution plan in embedded SQL in PL/pgSQL is reused for more calls - and it is optimized for more often value (not for really used value). Sometimes this difference can make really big slowdowns.
Then you can use dynamic SQL - EXECUTE statement. Dynamic SQL uses only once executed plans and it uses real parameters. It should to fix this issue.
Example of embedded SQL with reused prepared plans.
CREATE OR REPLACE FUNCTION fx1(_surname text)
RETURNS int AS $$
BEGIN
RETURN (SELECT count(*) FROM people WHERE surname = _surname)
END;
Example with dynamic SQL:
CREATE OR REPLACE FUNCTION fx2(_surname text)
RETURNS int AS $$
DECLARE result int;
BEGIN
EXECUTE 'SELECT count(*) FROM people WHERE surname = $1' INTO result
USING _surname;
RETURN result;
END;
$$ LANGUAGE plpgsql;
Second function can be faster if your dataset contains some terrible often surname - then common plan will be seq scan, but lot of time you will ask some other surname, and you will want to use index scan. Dynamical query parametrization (like ($1 IS NULL OR $1 = '''' OR rr.code = $1 OR) has same effect.
Your queries are not the same.
The first one has
WHERE cc.employee_id = contract.employee_id
where the second one has:
WHERE employee_id = he.id
And also:
($1 IS NULL OR $1 = '''' OR rr.code = $1 OR
he.login = $1)
Please test again with identical queries and identical values.

How to pass timestamps to a function and retrieve resulting rows

I have created custom data type. In that I have given alias name of the one field. you will get that in body of the function below.
create type voucher as (
ori numeric, RECEIPT_NO numeric
, receipt_date timestamp with time zone, reg_no character varying
, patient_name character varying, tot_refund_bill_amount double precision
, username character varying );
Thea above statement completes successfully.
Then I want to create a function:
create or replace function billing.voucher_receipt (in_from_date timestamp with time zone, in_to_date timestamp with time zone)
returns setof voucher as $$
declare
out_put voucher%rowtype;
begin
return query(select C.receipt_no as ori ,A.RECEIPT_NO, receipt_date , A.reg_no, patient_name, tot_refund_bill_amount, username
from billing.tran_counter_receipt as a inner join mas_user as b on a.ent_by=b.uid AND cash_book='REFUND'
INNER JOIN billing.tran_BILL AS C ON C.REG_NO=A.REG_NO AND C.CASH_BOOK='GCASH' where receipt_date>=in_from_date and receipt_date<=in_to_date);
end;$$
LANGUAGE plpgsql
Executes without problem.
But when I call it with input like this:
select * from voucher_receipt ('2014-09-25 11:42:44.298346+05:30'
, '2014-09-29 11:03:47.573049+05:30')
it shows an error:
ERROR: function voucher_receipt(unknown, unknown) does not exist
LINE 1: select * from voucher_receipt ('2014-09-25 11:42:44.298346+0...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
Can any one help me out from this?
Explain error
You created your function in the schema billing with:
create or replace function billing.voucher_receipt( ...
Then you call without schema-qualification:
select * from voucher_receipt ( ...
This only works while your current setting for search_path includes the schema billing.
Better function
You don't need to create a composite type. Unless you need the same type in multiple places just use RETURNS TABLE to define the return type in the function:
CREATE OR REPLACE FUNCTION billing.voucher_receipt (_from timestamptz
, _to timestamptz)
RETURNS TABLE (
ori numeric
, receipt_no numeric
, receipt_date timestamptz
, reg_no varchar
, patient_name varchar
, tot_refund_bill_amount float8
, username varchar) AS
$func$
BEGIN
RETURN QUERY
SELECT b.receipt_no -- AS ori
, cr.RECEIPT_NO
, ??.receipt_date
, cr.reg_no
, ??.patient_name
, ??.tot_refund_bill_amount
, ??.username
FROM billing.tran_counter_receipt cr
JOIN billing.tran_bill b USING (reg_no)
JOIN mas_user u ON u.uid = cr.ent_by
WHERE ??.receipt_date >= _from
AND ??.receipt_date <= _to
AND b.CASH_BOOK = 'GCASH'
AND ??.cash_book = 'REFUND'
END
$func$ LANGUAGE plpgsql;
Notes
Don't call your parameters "date" while they are actually timestamptz.
RETURN QUERY does not require parentheses.
No need for DECLARE out_put voucher%rowtype; at all.
Your format was inconsistent and messy. That ruins readability and that's also where bugs can hide.
This could just as well be a simple SQL function.
Column names in RETURNS TABLE are visible in the function body almost everywhere. table-qualify columns in your query to avoid ambiguities (and errors). Replace all ??. I left in the code, where information was missing.
Output column names are superseded by names in the RETURNS declaration. So AS ori in the SELECT list is just documentation in this case.
Why schema-qualify billing.tran_bill but not mas_user?