I need a function created (EscapeAmpersand). The function should take 1 varchar(255) parameter. It should return a varchar(255) parameter. The purpose of the function is to take the input, replace ‘&’ with ‘&’ and return the results.
Examples:
Input: ‘my dog & me’ return: ‘my dog & me’
Input: ‘my dog and me’ return: ‘my dog and me’
Scalar-valued function:
CREATE FUNCTION fReplaceAmpersand (#InputString VARCHAR(255))
RETURNS VARCHAR(1000)
AS
BEGIN
RETURN REPLACE(#InputString, '&', '&')
END
GO
Usage:
SELECT dbo.fReplaceAmpersand(MyText)
FROM MyTable
From a performance standpoint, if your tables are very large you'll be better off using a table-valued function and OUTER APPLY:
CREATE FUNCTION fReplaceAmpersand (#InputString VARCHAR(255))
RETURNS TABLE
AS
RETURN
SELECT REPLACE(#InputString, '&', '&') AS OutputString
GO
-- ----------------------------------------------------
SELECT a.OutputString
FROM MyTable m
OUTER APPLY dbo.fReplaceAmpersand(m.MyText) a
Non-procedural table-valued functions can operate several times faster than scalar-valued ones.
REPLACE( input,'&','&'||'amp')
Related
I implemented this function in my Postgres database: http://www.cureffi.org/2013/03/19/automatically-creating-pivot-table-column-names-in-postgresql/
Here's the function:
create or replace function xtab (tablename varchar, rowc varchar, colc varchar, cellc varchar, celldatatype varchar) returns varchar language plpgsql as $$
declare
dynsql1 varchar;
dynsql2 varchar;
columnlist varchar;
begin
-- 1. retrieve list of column names.
dynsql1 = 'select string_agg(distinct '||colc||'||'' '||celldatatype||''','','' order by '||colc||'||'' '||celldatatype||''') from '||tablename||';';
execute dynsql1 into columnlist;
-- 2. set up the crosstab query
dynsql2 = 'select * from crosstab (
''select '||rowc||','||colc||','||cellc||' from '||tablename||' group by 1,2 order by 1,2'',
''select distinct '||colc||' from '||tablename||' order by 1''
)
as ct (
'||rowc||' varchar,'||columnlist||'
);';
return dynsql2;
end
$$;
So now I can call the function:
select xtab('globalpayments','month','currency','(sum(total_fees)/sum(txn_amount)*100)::decimal(48,2)','text');
Which returns (because the return type of the function is varchar):
select * from crosstab (
'select month,currency,(sum(total_fees)/sum(txn_amount)*100)::decimal(48,2)
from globalpayments
group by 1,2
order by 1,2'
, 'select distinct currency
from globalpayments
order by 1'
) as ct ( month varchar,CAD text,EUR text,GBP text,USD text );
How can I get this function to not only generate the code for the dynamic crosstab, but also execute the result? I.e., the result when I manually copy/paste/execute is this. But I want it to execute without that extra step: the function shall assemble the dynamic query and execute it:
Edit 1
This function comes close, but I need it to return more than just the first column of the first record
Taken from: Are there any way to execute a query inside the string value (like eval) in PostgreSQL?
create or replace function eval( sql text ) returns text as $$
declare
as_txt text;
begin
if sql is null then return null ; end if ;
execute sql into as_txt ;
return as_txt ;
end;
$$ language plpgsql
usage: select * from eval($$select * from analytics limit 1$$)
However it just returns the first column of the first record :
eval
----
2015
when the actual result looks like this:
Year, Month, Date, TPV_USD
---- ----- ------ --------
2016, 3, 2016-03-31, 100000
What you ask for is impossible. SQL is a strictly typed language. PostgreSQL functions need to declare a return type (RETURNS ..) at the time of creation.
A limited way around this is with polymorphic functions. If you can provide the return type at the time of the function call. But that's not evident from your question.
Refactor a PL/pgSQL function to return the output of various SELECT queries
You can return a completely dynamic result with anonymous records. But then you are required to provide a column definition list with every call. And how do you know about the returned columns? Catch 22.
There are various workarounds, depending on what you need or can work with. Since all your data columns seem to share the same data type, I suggest to return an array: text[]. Or you could return a document type like hstore or json. Related:
Dynamic alternative to pivot with CASE and GROUP BY
Dynamically convert hstore keys into columns for an unknown set of keys
But it might be simpler to just use two calls: 1: Let Postgres build the query. 2: Execute and retrieve returned rows.
Selecting multiple max() values using a single SQL statement
I would not use the function from Eric Minikel as presented in your question at all. It is not safe against SQL injection by way of maliciously malformed identifiers. Use format() to build query strings unless you are running an outdated version older than Postgres 9.1.
A shorter and cleaner implementation could look like this:
CREATE OR REPLACE FUNCTION xtab(_tbl regclass, _row text, _cat text
, _expr text -- still vulnerable to SQL injection!
, _type regtype)
RETURNS text
LANGUAGE plpgsql AS
$func$
DECLARE
_cat_list text;
_col_list text;
BEGIN
-- generate categories for xtab param and col definition list
EXECUTE format(
$$SELECT string_agg(quote_literal(x.cat), '), (')
, string_agg(quote_ident (x.cat), %L)
FROM (SELECT DISTINCT %I AS cat FROM %s ORDER BY 1) x$$
, ' ' || _type || ', ', _cat, _tbl)
INTO _cat_list, _col_list;
-- generate query string
RETURN format(
'SELECT * FROM crosstab(
$q$SELECT %I, %I, %s
FROM %I
GROUP BY 1, 2 -- only works if the 3rd column is an aggregate expression
ORDER BY 1, 2$q$
, $c$VALUES (%5$s)$c$
) ct(%1$I text, %6$s %7$s)'
, _row, _cat, _expr -- expr must be an aggregate expression!
, _tbl, _cat_list, _col_list, _type);
END
$func$;
Same function call as your original version. The function crosstab() is provided by the additional module tablefunc which has to be installed. Basics:
PostgreSQL Crosstab Query
This handles column and table names safely. Note the use of object identifier types regclass and regtype. Also works for schema-qualified names.
Table name as a PostgreSQL function parameter
However, it is not completely safe while you pass a string to be executed as expression (_expr - cellc in your original query). This kind of input is inherently unsafe against SQL injection and should never be exposed to the general public.
SQL injection in Postgres functions vs prepared queries
Scans the table only once for both lists of categories and should be a bit faster.
Still can't return completely dynamic row types since that's strictly not possible.
Not quite impossible, you can still execute it (from a query execute the string and return SETOF RECORD.
Then you have to specify the return record format. The reason in this case is that the planner needs to know the return format before it can make certain decisions (materialization comes to mind).
So in this case you would EXECUTE the query, return the rows and return SETOF RECORD.
For example, we could do something like this with a wrapper function but the same logic could be folded into your function:
CREATE OR REPLACE FUNCTION crosstab_wrapper
(tablename varchar, rowc varchar, colc varchar,
cellc varchar, celldatatype varchar)
returns setof record language plpgsql as $$
DECLARE outrow record;
BEGIN
FOR outrow IN EXECUTE xtab($1, $2, $3, $4, $5)
LOOP
RETURN NEXT outrow
END LOOP;
END;
$$;
Then you supply the record structure on calling the function just like you do with crosstab.
Then when you all the query you would have to supply a record structure (as (col1 type, col2 type, etc) like you do with connectby.
As we know, plpgsql functions can return a table like this:
RETURNS table(int, char(1), ...)
But how to write this function, when the list of columns is uncertain at the time of creating the function.
When a function returns anonymous records
RETURNS SETOF record
you have to provide a column definition list when calling it with SELECT * FROM. SQL demands to know column names and types to interpret *. For registered tables and types this is provided by the system catalog. For functions you need to declare it yourself one way or the other. Either in the function definition or in the call. The call could look like #Craig already provided. You probably didn't read his answer carefully enough.
Depending on what you need exactly, there are a number of ways around this, though:
1) Return a single anonymous record
Example:
CREATE OR REPLACE FUNCTION myfunc_single() -- return a single anon rec
RETURNS record AS
$func$
DECLARE
rec record;
BEGIN
SELECT into rec 1, 'foo'; -- note missing type for 'foo'
RETURN rec;
END
$func$ LANGUAGE plpgsql;
This is a very limited niche. Only works for a single anonymous record from a function defined with:
RETURNS record
Call without * FROM:
SELECT myfunc_single();
Won't work for a SRF (set-returning function) and only returns a string representation of the whole record (type record). Rarely useful.
To get individual cols from a single anonymous record, you need to provide a column definition list again:
SELECT * FROM myfunc_single() AS (id int, txt unknown); -- note "unknown" type
2) Return well known row type with a super-set of columns
Example:
CREATE TABLE t (id int, txt text, the_date date);
INSERT INTO t VALUES (3, 'foz', '2014-01-13'), (4, 'baz', '2014-01-14');
CREATE OR REPLACE FUNCTION myfunc_tbl() -- return well known table
RETURNS SETOF t AS
$func$
BEGIN
RETURN QUERY
TABLE t;
-- SELECT * FROM t; -- equivalent
END
$func$ LANGUAGE plpgsql;
The function returns all columns of the table. This is short and simple and performance won't suffer as long as your table doesn't hold a huge number of columns or huge columns.
Select individual columns on call:
SELECT id, txt FROM myfunc_tbl();
SELECT id, the_date FROM myfunc_tbl();
-> SQLfiddle demonstrating all.
3) Advanced solutions
This answer is long enough already. And this closely related answer has it all:
Refactor a PL/pgSQL function to return the output of various SELECT queries
Look to the last chapter in particular: Various complete table types
If the result is of uncertain/undefined format you must use RETURNS record or (for a multi-row result) RETURNS SETOF record.
The calling function must then specify the table format, eg:
SELECT my_func() AS result(a integer, b char(1));
BTW, char is an awful data type with insane space-padding rules that date back to the days of fixed-width file formats. Don't use it. Always just use text or varchar.
Given comments, let's make this really explicit:
regress=> CREATE OR REPLACE FUNCTION f_something() RETURNS SETOF record AS $$
SELECT 1, 2, TEXT 'a';
$$ LANGUAGE SQL;
CREATE FUNCTION
regress=> SELECT * FROM f_something();
ERROR: a column definition list is required for functions returning "record"
LINE 1: SELECT * FROM f_something();
regress=> SELECT * FROM f_something() AS x(a integer, b integer, c text);
a | b | c
---+---+---
1 | 2 | a
(1 row)
I have a situation where I want to return the join between two views. and that's a lot of columns. It was pretty easy in sql server. But in PostgreSQL when I do the join. I get the error "a column definition list is required".
Is there any way I can bypass this, I don't want to provide the definitions of returning columns.
CREATE OR REPLACE FUNCTION functionA(username character varying DEFAULT ''::character varying, databaseobject character varying DEFAULT ''::character varying)
RETURNS SETOF ???? AS
$BODY$
Declare
SqlString varchar(4000) = '';
BEGIN
IF(UserName = '*') THEN
Begin
SqlString := 'select * from view1 left join ' + databaseobject + ' as view2 on view1.id = view2.id';
End;
ELSE
Begin
SqlString := 'select * from view3 left join ' + databaseobject + ' as view2 on view3.id = view2.id';
End;
END IF;
execute (SqlString );
END;
$BODY$
Sanitize function
What you currently have can be simplified / sanitized to:
CREATE OR REPLACE FUNCTION func_a (username text = '', databaseobject text = '')
RETURNS ????
LANGUAGE plpgsql AS
$func$
BEGIN
RETURN QUERY EXECUTE
format ('SELECT * FROM %s v1 LEFT JOIN %I v2 USING (id)'
, CASE WHEN username = '*' THEN 'view1' ELSE 'view3' END
, databaseobject);
END
$func$;
You only need additional instances of BEGIN ... END in the function body to start separate code blocks with their own scope, which is rarely needed.
The standard SQL concatenation operator is ||. + is a "creative" addition of your former vendor.
Don't use CaMeL-case identifiers unless you double-quote them. Best don't use them at all See:
Are PostgreSQL column names case-sensitive?
varchar(4000) is also tailored to a specific limitation of SQL Server. It has no specific significance in Postgres. Only use varchar(4000) if you actually need a limit of 4000 characters. I would just use text - except that we don't need any variables at all here, after simplifying the function.
If you have not used format(), yet, consult the manual here.
Return type
Now, for your actual question: The return type for a dynamic query can be tricky since SQL requires that to be declared at call time at the latest. If you have a table or view or composite type in your database already matching the column definition list, you can just use that:
CREATE FUNCTION foo()
RETURNS SETOF my_view AS
...
Else, spell the column definition list with out with (simplest) RETURNS TABLE:
CREATE FUNCTION foo()
RETURNS TABLE (col1 int, col2 text, ...) AS
...
If you are making the row type up as you go, you can return anonymous records:
CREATE FUNCTION foo()
RETURNS SETOF record AS
...
But then you have to provide a column definition list with every call, so I hardly ever use that.
I wouldn't use SELECT * to begin with. Use a definitive list of columns to return and declare your return type accordingly:
CREATE OR REPLACE FUNCTION func_a(username text = '', databaseobject text = '')
RETURNS TABLE(col1 int, col2 text, col3 date)
LANGUAGE plpgsql AS
$func$
BEGIN
RETURN QUERY EXECUTE
format ($f$SELECT v1.col1, v1.col2, v2.col3
FROM %s v1 LEFT JOIN %I v2 USING (id)$f$
, CASE WHEN username = '*' THEN 'view1' ELSE 'view3' END
, databaseobject);
END
$func$;
For completely dynamic queries, consider building the query in your client to begin with, instead of using a function.
You need to understand basics first:
Refactor a PL/pgSQL function to return the output of various SELECT queries
PL/pgSQL in the Postgres manual
Then there are more advanced options with polymorphic types, which allow you to pass the return type at call time. More in the last chapter of:
Refactor a PL/pgSQL function to return the output of various SELECT queries
Can functions be stored as anonymous functions directly in column as its value?
Let's say I want this function be stored in column.
Example (pseudocode):
Table my_table: pk (int), my_function (func)
func ( x ) { return x * 100 }
And later use it as:
select
t.my_function(some_input) AS output
from
my_table as t
where t.pk = 1999
Function may vary for each pk.
Your title asks something else than your example.
A function has to be created before you can call it. (title)
An expression has to be evaluated. You would need a meta-function for that. (example)
Here are solutions for both:
1. Evaluate expressions dynamically
You have to take into account that the resulting type can vary. I use polymorphic types for that.
CREATE OR REPLACE FUNCTION f1(int)
RETURNS int
LANGUAGE sql IMMUTABLE AS
'SELECT $1 * 100;';
CREATE OR REPLACE FUNCTION f2(text)
RETURNS text
LANGUAGE sql IMMUTABLE AS
$$SELECT $1 || '_foo';$$;
CREATE TABLE my_expr (
expr text PRIMARY KEY
, def text
, rettype regtype
);
INSERT INTO my_expr VALUES
('x', 'f1(3)' , 'int')
, ('y', $$f2('bar')$$, 'text')
, ('z', 'now()' , 'timestamptz')
;
CREATE OR REPLACE FUNCTION f_eval(text, _type anyelement = 'NULL'::text, OUT _result anyelement)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE
'SELECT ' || (SELECT def FROM my_expr WHERE expr = $1)
INTO _result;
END
$func$;
Related:
Refactor a PL/pgSQL function to return the output of various SELECT queries
Call:
SQL is strictly typed, the same result column can only have one data type. For multiple rows with possibly heterogeneous data types, you might settle for type text, as every data type can be cast to and from text:
SELECT *, f_eval(expr) AS result -- default to type text
FROM my_expr;
Or return multplce columns like:
SELECT *
, CASE WHEN rettype = 'text'::regtype THEN f_eval(expr) END AS text_result -- default to type text
, CASE WHEN rettype = 'int'::regtype THEN f_eval(expr, NULL::int) END AS int_result
, CASE WHEN rettype = 'timestamptz'::regtype THEN f_eval(expr, NULL::timestamptz) END AS tstz_result
-- , more?
FROM my_expr;
db<>fiddle here
2. Create and use functions dynamically
It is possible to create functions dynamically and then use them. You cannot do that with plain SQL, however. You will have to use another function to do that or at least an anonymous code block (DO statement), introduced in PostgreSQL 9.0.
It can work like this:
CREATE TABLE my_func (func text PRIMARY KEY, def text);
INSERT INTO my_func VALUES
('f'
, $$CREATE OR REPLACE FUNCTION f(int)
RETURNS int
LANGUAGE sql IMMUTABLE AS
'SELECT $1 * 100;'$$);
CREATE OR REPLACE FUNCTION f_create_func(text)
RETURNS void
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE (SELECT def FROM my_func WHERE func = $1);
END
$func$;
Call:
SELECT f_create_func('f');
SELECT f(3);
db<>fiddle here
You may want to drop the function afterwards.
In most cases you should just create the functions instead and be done with it. Use separate schemas if you have problems with multiple versions or privileges.
For more information on the features I used here, see my related answer on dba.stackexchange.com.
FYI: I am completely new to using cursors...
So I have one function that is a cursor:
CREATE FUNCTION get_all_product_promos(refcursor, cursor_object_id integer) RETURNS refcursor AS '
BEGIN
OPEN $1 FOR SELECT *
FROM promos prom1
JOIN promo_objects ON (prom1.promo_id = promo_objects.promotion_id)
WHERE prom1.active = true AND now() BETWEEN prom1.start_date AND prom1.end_date
AND promo_objects.object_id = cursor_object_id
UNION
SELECT prom2.promo_id
FROM promos prom2
JOIN promo_buy_objects ON (prom2.promo_id =
promo_buy_objects.promo_id)
LEFT JOIN promo_get_objects ON prom2.promo_id = promo_get_objects.promo_id
WHERE (prom2.buy_quantity IS NOT NULL OR prom2.buy_quantity > 0) AND
prom2.active = true AND now() BETWEEN prom2.start_date AND
prom2.end_date AND promo_buy_objects.object_id = cursor_object_id;
RETURN $1;
END;
' LANGUAGE plpgsql;
SO then in another function I call it and need to process it:
...
--Get the promotions from the cursor
SELECT get_all_product_promos('promo_cursor', this_object_id)
updated := FALSE;
IF FOUND THEN
--Then loop through your results
LOOP
FETCH promo_cursor into this_promotion
--Preform comparison logic -this is necessary as this logic is used in other contexts from other functions
SELECT * INTO best_promo_results FROM get_best_product_promos(this_promotion, this_object_id, get_free_promotion, get_free_promotion_value, current_promotion_value, current_promotion);
...
SO the idea here is to select from the cursor, loop using fetch (next is assumed correct?) and put the record fetched into this_promotion. Then send the record in this_promotion to another function. I can't figure out what to declare the type of this_promotion in get_best_product_promos. Here is what I have:
CREATE OR REPLACE FUNCTION get_best_product_promos(this_promotion record, this_object_id integer, get_free_promotion integer, get_free_promotion_value numeric(10,2), current_promotion_value numeric(10,2), current_promotion integer)
RETURNS...
It tells me: ERROR: plpgsql functions cannot take type record
OK first I tried:
CREATE OR REPLACE FUNCTION get_best_product_promos(this_promotion get_all_product_promos, this_object_id integer, get_free_promotion integer, get_free_promotion_value numeric(10,2), current_promotion_value numeric(10,2), current_promotion integer)
RETURNS...
Because I saw some syntax in the Postgres docs showed a function being created w/ a input parameter that had a type 'tablename' this works, but it has to be a tablename not a function :( I know I am so close, I was told to use cursors to pass records around. So I studied up. Please help.
One possibility would be to define the query you have in get_all_product_promos as a all_product_promos view instead. Then you would automatically have the "all_product_promos%rowtype" type to pass between functions.
That is, something like:
CREATE VIEW all_product_promos AS
SELECT promo_objects.object_id, prom1.*
FROM promos prom1
JOIN promo_objects ON (prom1.promo_id = promo_objects.promotion_id)
WHERE prom1.active = true AND now() BETWEEN prom1.start_date AND prom1.end_date
UNION ALL
SELECT promo_buy_objects.object_id, prom2.*
FROM promos prom2
JOIN promo_buy_objects ON (prom2.promo_id = promo_buy_objects.promo_id)
LEFT JOIN promo_get_objects ON prom2.promo_id = promo_get_objects.promo_id
WHERE (prom2.buy_quantity IS NOT NULL OR prom2.buy_quantity > 0)
AND prom2.active = true
AND now() BETWEEN prom2.start_date AND prom2.end_date
You should be able to verify using EXPLAIN that querying SELECT * FROM all_product_promos WHERE object_id = ? takes the object_id parameter into the two subqueries rather than filtering afterwards. Then from another function you can write:
DECLARE
this_promotion all_product_promos%ROWTYPE;
BEGIN
FOR this_promotion IN
SELECT * FROM all_product_promos WHERE object_id = this_object_id
LOOP
-- deal with promotion in this_promotion
END LOOP;
END
TBH I would avoid using cursors to pass records around in PLPGSQL. In fact, I would avoid using cursors in PLPGSQL full stop- unless you need to pass a whole resultset to another function for some reason. This method of simply looping through the statement is much simpler, with the caveat that the entire resultset is materialised into memory first.
The other disadvantage with this approach is that if you need to add a column to all_product_promos, you will need to recreate all the functions that depend on it, since you can't add columns to a view with 'alter view'. AFAICT this affects named types create with CREATE TYPE too, since ALTER TYPE doesn't seem to allow you to add columns to a type either.
So, you can specify a record format using "CREATE TYPE" to pass between functions. Any relation automagically specifies a type called <relation>%ROWTYPE which you can also use.
The answer:
Select specific fields in the cursor function instead of *
then:
CREATE TYPE get_all_product_promos as (buy_quantity integer, discount_amount numeric(10,2), get_quantity integer, discount_type integer, promo_id integer);
Then I can say:
CREATE OR REPLACE FUNCTION get_best_product_promos(this_promotion get_all_product_promos, this_object_id integer, get_free_promotion integer, get_free_promotion_value numeric(10,2), current_promotion_value numeric(10,2), current_promotion integer)
RETURNS...