I have the following query that I call from .Net core
select * from crosstab(
$$select
cast(s.sample_name as text) as sample_name,
cast(dt.demographic_type_name as text) as demographic_type_name,
cast(sdd.demographic_value as text) as demographic_value
from
sample s
inner join sample_demographic_data sdd on s.sample_id = sdd.sample_id
inner join demographic_type dt on sdd.demographic_type_id = dt.demographic_type_id and dt.sort_order >= 0
where
s.sample_id in (0,14,28)
$$
) as t (
sample_name text,
"SEX" text,
"TUMOR_STAGE" text,
"RACE" text,
"TUMOR_TYPE" text,
"PATIENT_AGE" text);
);
This produces the following output:
However, I'd like to pass in an array of int for the sample Ids. I changed the where clause to look like this:
s.sample_id = ANY(#sample_id)
and added code from the .Net core application to add the parameter:
cmd.Parameters.Add("#sample_id", NpgsqlDbType.Array | NpgsqlDbType.Integer).Value = sampleIds;
When I do that, I get an error:
Npgsql.PostgresException (0x80004005): 42702: column reference "sample_id" is ambiguous
My assumption is that the parameter is not playing nicely with the $$ quoted query for crosstab. What is the correct syntax for including a parameter into the $$ quoted area of a crosstab query?
Related
I've problem when try to cast data type from TEXT to BIGINT when using WHERE IN on PostgreSQL in procedure. This always gives
operator does not exist: bigint = text. Try cast the variable in the query.
But still get the same notice. This is example query:
DECLARE
-- $1 params text
BEGIN
SELECT * FROM table_a where
colId IN($1); // notice is here, colId is bigint
END
/*Call the procedure*/
SELECT my_function('1,2,3,4,5');
How do we cast the variable? Thanks!
Using strings for id list is wrong design. You can use a arrays in PostgreSQL.
For example
CREATE OR REPLACE FUNCTION foo(VARIADIC ids int[])
RETURNS SETOF table_a AS $$
SELECT * FROM table_a WHERE id = ANY($1)
$$ LANGUAGE sql;
SELECT foo(1,2,3);
But, usually wrapping simple SQL to functions looks like broken design. Procedures should not to replace views.
I implemented this function in my Postgres database: http://www.cureffi.org/2013/03/19/automatically-creating-pivot-table-column-names-in-postgresql/
Here's the function:
create or replace function xtab (tablename varchar, rowc varchar, colc varchar, cellc varchar, celldatatype varchar) returns varchar language plpgsql as $$
declare
dynsql1 varchar;
dynsql2 varchar;
columnlist varchar;
begin
-- 1. retrieve list of column names.
dynsql1 = 'select string_agg(distinct '||colc||'||'' '||celldatatype||''','','' order by '||colc||'||'' '||celldatatype||''') from '||tablename||';';
execute dynsql1 into columnlist;
-- 2. set up the crosstab query
dynsql2 = 'select * from crosstab (
''select '||rowc||','||colc||','||cellc||' from '||tablename||' group by 1,2 order by 1,2'',
''select distinct '||colc||' from '||tablename||' order by 1''
)
as ct (
'||rowc||' varchar,'||columnlist||'
);';
return dynsql2;
end
$$;
So now I can call the function:
select xtab('globalpayments','month','currency','(sum(total_fees)/sum(txn_amount)*100)::decimal(48,2)','text');
Which returns (because the return type of the function is varchar):
select * from crosstab (
'select month,currency,(sum(total_fees)/sum(txn_amount)*100)::decimal(48,2)
from globalpayments
group by 1,2
order by 1,2'
, 'select distinct currency
from globalpayments
order by 1'
) as ct ( month varchar,CAD text,EUR text,GBP text,USD text );
How can I get this function to not only generate the code for the dynamic crosstab, but also execute the result? I.e., the result when I manually copy/paste/execute is this. But I want it to execute without that extra step: the function shall assemble the dynamic query and execute it:
Edit 1
This function comes close, but I need it to return more than just the first column of the first record
Taken from: Are there any way to execute a query inside the string value (like eval) in PostgreSQL?
create or replace function eval( sql text ) returns text as $$
declare
as_txt text;
begin
if sql is null then return null ; end if ;
execute sql into as_txt ;
return as_txt ;
end;
$$ language plpgsql
usage: select * from eval($$select * from analytics limit 1$$)
However it just returns the first column of the first record :
eval
----
2015
when the actual result looks like this:
Year, Month, Date, TPV_USD
---- ----- ------ --------
2016, 3, 2016-03-31, 100000
What you ask for is impossible. SQL is a strictly typed language. PostgreSQL functions need to declare a return type (RETURNS ..) at the time of creation.
A limited way around this is with polymorphic functions. If you can provide the return type at the time of the function call. But that's not evident from your question.
Refactor a PL/pgSQL function to return the output of various SELECT queries
You can return a completely dynamic result with anonymous records. But then you are required to provide a column definition list with every call. And how do you know about the returned columns? Catch 22.
There are various workarounds, depending on what you need or can work with. Since all your data columns seem to share the same data type, I suggest to return an array: text[]. Or you could return a document type like hstore or json. Related:
Dynamic alternative to pivot with CASE and GROUP BY
Dynamically convert hstore keys into columns for an unknown set of keys
But it might be simpler to just use two calls: 1: Let Postgres build the query. 2: Execute and retrieve returned rows.
Selecting multiple max() values using a single SQL statement
I would not use the function from Eric Minikel as presented in your question at all. It is not safe against SQL injection by way of maliciously malformed identifiers. Use format() to build query strings unless you are running an outdated version older than Postgres 9.1.
A shorter and cleaner implementation could look like this:
CREATE OR REPLACE FUNCTION xtab(_tbl regclass, _row text, _cat text
, _expr text -- still vulnerable to SQL injection!
, _type regtype)
RETURNS text
LANGUAGE plpgsql AS
$func$
DECLARE
_cat_list text;
_col_list text;
BEGIN
-- generate categories for xtab param and col definition list
EXECUTE format(
$$SELECT string_agg(quote_literal(x.cat), '), (')
, string_agg(quote_ident (x.cat), %L)
FROM (SELECT DISTINCT %I AS cat FROM %s ORDER BY 1) x$$
, ' ' || _type || ', ', _cat, _tbl)
INTO _cat_list, _col_list;
-- generate query string
RETURN format(
'SELECT * FROM crosstab(
$q$SELECT %I, %I, %s
FROM %I
GROUP BY 1, 2 -- only works if the 3rd column is an aggregate expression
ORDER BY 1, 2$q$
, $c$VALUES (%5$s)$c$
) ct(%1$I text, %6$s %7$s)'
, _row, _cat, _expr -- expr must be an aggregate expression!
, _tbl, _cat_list, _col_list, _type);
END
$func$;
Same function call as your original version. The function crosstab() is provided by the additional module tablefunc which has to be installed. Basics:
PostgreSQL Crosstab Query
This handles column and table names safely. Note the use of object identifier types regclass and regtype. Also works for schema-qualified names.
Table name as a PostgreSQL function parameter
However, it is not completely safe while you pass a string to be executed as expression (_expr - cellc in your original query). This kind of input is inherently unsafe against SQL injection and should never be exposed to the general public.
SQL injection in Postgres functions vs prepared queries
Scans the table only once for both lists of categories and should be a bit faster.
Still can't return completely dynamic row types since that's strictly not possible.
Not quite impossible, you can still execute it (from a query execute the string and return SETOF RECORD.
Then you have to specify the return record format. The reason in this case is that the planner needs to know the return format before it can make certain decisions (materialization comes to mind).
So in this case you would EXECUTE the query, return the rows and return SETOF RECORD.
For example, we could do something like this with a wrapper function but the same logic could be folded into your function:
CREATE OR REPLACE FUNCTION crosstab_wrapper
(tablename varchar, rowc varchar, colc varchar,
cellc varchar, celldatatype varchar)
returns setof record language plpgsql as $$
DECLARE outrow record;
BEGIN
FOR outrow IN EXECUTE xtab($1, $2, $3, $4, $5)
LOOP
RETURN NEXT outrow
END LOOP;
END;
$$;
Then you supply the record structure on calling the function just like you do with crosstab.
Then when you all the query you would have to supply a record structure (as (col1 type, col2 type, etc) like you do with connectby.
I am using postgres functions to create a crosstab table.
My first dynamic query function (dynamic_crosstab) creates a sql select statement, and then this select statement gives the final result on execution.
dynamic_crosstab functions works perfectly
I need to execute this select query (result of dynamic_crosstab function) by using parameters, so I am again using a function again.
CREATE OR REPLACE FUNCTION primary_cross(
date,
date,
integer)
RETURNS text AS
$BODY$
declare
val_1 text;
begin
select * from dynamic_crosstab('select
p.location_id, p.employee_id, pt.description, sum(p.hours_allocated) as hours_allocated
from
preference_type pt, preference p, preference_date_etl pde, date_etl de
where
pt.id = p.preference_type_id and
pde.preference_id = p.id and
pde.corporation_id = $3 and
de.id = pde.date_etl_id and
pde.deleted = ''''N'''' and
p.deleted = ''''N'''' and
pt.deleted = ''''N'''' and
de.local_date between ''''$1'''' and ''''$2'''' and
p.employee_id IN (
select id from employee where user_id IN (select id from app_user where corporation_id = $3))
group by p.location_id, p.employee_id, pt.description',
'select distinct description from preference_type where corporation_id =$3',
'text','location_id int , employee_id int',false) INTO val_1;
return val_1;
end;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
Note that dynamic_crosstab function uses 4 input parameters. 2 sql queries, 2 text value and a boolean.
Now the problem is that parameters are passed correctly to first input of dynamic_crosstab function but they do not pass to the second input of dynamic_crosstab which is:
'select distinct description from preference_type where deleted=''''N'''' and corporation_id =$3'
I get the following error:
ERROR: there is no parameter $3
LINE 1: ...ct description from preference_type where corporation_id =$3
^
QUERY: select distinct description from preference_type where corporation_id =$3
Please advise how i can use the parameters to make this work.
Thank You.
I have a table in PostgreSQL for storing details of films,
film_id serial NOT NULL,
title text,
language text,
directors text[]
I can write a query to find all films having a director name 'ab'
SELECT * FROM films where 'ab' like any(films.directors)
But How can I write a query to search the table to get all films having a director name starting with 'ab',this query does not giving me any result
SELECT * FROM films where 'ab%' like any(films.directors)
I can think of two different ways (none of them really elegant):
Unnest the array and then search on the elements using LIKE, if you need the result as an array again, you need to aggregate (which means adding more columns than just the directors and the film_id is going to be complicated:
select film_id, array_agg(name) as directors
from (
SELECT film_id, unnest(directors) as name
FROM films
) t
where name like 'ab%'
group by film_id;
Another option (equally slow) might be to create a long string with a delimiter out of the array and then search inside that string:
select *
from films
where '###'||array_to_string(directors, '###') like '%###ab%'
Prepending the delimiter to the string ensures that every entry starts with the delimiter and the condition like '%###ab%' will only match names that actually start with ab.
There is another way too. The reason, your original plan could not work, is because the array comparison expressions can only use an operator & an array expression on the right side of the operator:
<expression> operator ANY|SOME|ALL (<array expression>)
And all the pattern matching operators work reversed (on the right side there are the pattern) -- you could use that too, but only if you have a pattern array, which you want to test again a single string. this is usually not so useful, but you can make some operators which work reversed:
CREATE OR REPLACE FUNCTION reverse_like(text, text) RETURNS BOOLEAN LANGUAGE SQL AS 'SELECT $2 LIKE $1';
CREATE OR REPLACE FUNCTION reverse_ilike(text, text) RETURNS BOOLEAN LANGUAGE SQL AS 'SELECT $2 ILIKE $1';
CREATE OR REPLACE FUNCTION reverse_similarto(text, text) RETURNS BOOLEAN LANGUAGE SQL AS 'SELECT $2 SIMILAR TO $1';
CREATE OR REPLACE FUNCTION reverse_posix_match(text, text) RETURNS BOOLEAN LANGUAGE SQL AS 'SELECT $2 ~ $1';
CREATE OR REPLACE FUNCTION reverse_posix_imatch(text, text) RETURNS BOOLEAN LANGUAGE SQL AS 'SELECT $2 ~* $1';
CREATE OPERATOR ~~> (PROCEDURE = reverse_like, LEFTARG = text, RIGHTARG = text);
CREATE OPERATOR ~~*> (PROCEDURE = reverse_ilike, LEFTARG = text, RIGHTARG = text);
CREATE OPERATOR ~~~> (PROCEDURE = reverse_similarto, LEFTARG = text, RIGHTARG = text);
CREATE OPERATOR ~> (PROCEDURE = reverse_posix_match, LEFTARG = text, RIGHTARG = text);
CREATE OPERATOR ~*> (PROCEDURE = reverse_posix_imatch, LEFTARG = text, RIGHTARG = text);
After that, you can use the ANY/SOME/ALL array comparisons:
SELECT *
FROM films
WHERE '^ab' ~> any(films.directors);
SELECT *
FROM films
WHERE 'ab%' ~~> any(films.directors);
SELECT *
FROM films
WHERE 'ab%' ~~~> any(films.directors);
SQLFiddle
SELECT *
FROM films
where directors like 'ab%';
should get you what you request. Notice how the expressions surronding the LIKE operator are switched from the order they were in.
Here is a link to the SELECT documentation for postgresql 9 which has an example of query using like as show above in my example.
ANY needs a subquery according to http://www.postgresql.org/docs/9.0/static/functions-subquery.html
Example: I am passing two parameters to function namely n(case number) and tname(table name), and want to display rows accordingly.
--Table "testing"
create table testing
(
rollno integer,
fname text,
lname text,
age integer,
branch text,
phno integer,
email text,
address text,
city text,
state text,
country text
)
--Rows insertion
insert into testing values(1,'aaa','bbb',25,'CS',1234567890,'abc#gmail.com','sector1','xyz','zyx','yxz');
insert into testing values(2,'zzz','xxx',25,'EE',987654321,'zzz#gmail.com','sector2','uvw','wvu','vuw');
--Function "f1()"
create or replace function f1(n integer,tname varchar)/*n for case number and tname for table name */
returns setof tname as
$body$
begin
case n
when 1 then
return query execute format ($$ select rollno,fname from %I $$,tname);
when 2 then
return query execute format ($$ select lname,age,branch from %I $$,tname);
when 3 then
return query execute format ($$ select phno,email,address,city,country from %I $$,tname);
end case;
end
$body$
language plpgsql;
--Function calling
select * from f1(1,'testing');
/*Show only case "1" select query result*/
select * from f1(2,'testing');
/*Show only case "2" select query result*/
select * from f1(3,'testing');
/*Show only case "3" select query result*/
While Craig is correct that return types cannot be dynamic in function declarations, there is a way around this with polymorphic types. This is surprisingly simple and would actually work flawlessly:
CREATE OR REPLACE FUNCTION data_of(_tbl_type anyelement)
RETURNS SETOF anyelement AS
$func$
BEGIN
RETURN QUERY EXECUTE 'SELECT * FROM '|| pg_typeof(_tbl_type);
END
$func$ LANGUAGE plpgsql;
Call (important!):
SELECT rollno,fname FROM data_of(NULL::testing);
SELECT * FROM data_of(NULL::my_schema.my_table);
SELECT * FROM data_of(NULL::my_custom_type);
What you need is a well-known type. For every table there is a well-known type automatically. But you can create any type, cast NULL to it and pass it to the function. This way you can build exactly what you have in your question ...
Related answer with a lot more details:
Refactor a PL/pgSQL function to return the output of various SELECT queries