I am trying to return the movie name and the number of cast and crew when given a text. When I input the string and am using ilike, my query returns no matching titles. I created a view previously that has the movie titles and the number of crew to be input in the function.
My code is:
create or replace view movies_crew as
select movies.id, movies.title, principals.role
from movies
join principals on principals.movie_id=movies.id
where principals.role <> 'producer'
;
create or replace view movie_makers as
select movies_crew.title, count(movies_crew.title) as ncrew
from movies_crew
where movies_crew.title = 'Fight Club'
group by movies_crew.title;
CREATE or REPLACE function Q11(partial_title text)
RETURNS SETOF text
AS $$
DECLARE
title text;
BEGIN
for title in
select movie_makers.title, movie_makers.ncrew
from movie_makers
where movie_makers.title ilike '%$1%'
loop
return next movie_makers.title||'has'||movie_makers.ncrew||'cast and crew';
end loop;
if(not found) then
return next 'No matching titles';
end if;
END;
$$ LANGUAGE plpgsql;
select * from q11('Fight Club')
My database is: https://drive.google.com/file/d/1NVRLiYBVbKuiazynx9Egav7c4_VHFEzP/view?usp=sharing
Your immediate quoting issue aside (has been addressed properly by Jeff), the function can be much simpler and faster like this:
CREATE or REPLACE FUNCTION q11(partial_title text)
RETURNS SETOF text
LANGUAGE plpgsql AS
$func$
BEGIN
RETURN QUERY
SELECT m.title || ' has ' || m.ncrew || ' cast and crew'
FROM movie_makers m
WHERE m.title ~* $1;
IF NOT FOUND THEN
RETURN NEXT 'No matching titles';
END IF;
END
$func$;
Major points:
Your function was still broken. References to movie_makers.title and movie_makers.ncrew wouldn't work that way. I fixed it.
Use RETURN QUERY instead of the loop. This way we also do not need to use or even declare any variables at all. See:
How to return result of a SELECT inside a function in PostgreSQL?
Optionally use the case insensitive regular expression match operator ~*. (Simpler, not faster.)
Difference between LIKE and ~ in Postgres
Either way, you may want to escape special characters. See:
Escape function for regular expression or LIKE patterns
Aside: hardly makes sense to filter on a view that already selects 'Fight Club' as its only row. For a meaningful search, you wouldn't use these views ...
ilike '%$1%'
$1 is not interpolated when inside single quotes, so you are searching for the literal characters $ and 1.
You could instead do:
ilike '%'||$1||'%'
Related
May I know on how to call an array in stored procedure? I tried to enclosed it with a bracket to put the column_name that need to be insert in the new table.
CREATE OR REPLACE PROCEDURE data_versioning_nonull(new_table_name VARCHAR(100),column_name VARCHAR(100)[], current_table_name VARCHAR(100))
language plpgsql
as $$
BEGIN
EXECUTE ('CREATE TABLE ' || quote_ident(new_table_name) || ' AS SELECT ' || quote_ident(column_name) || ' FROM ' || quote_ident(current_table_name));
END $$;
CALL data_versioning_nonull('sales_2019_sample', ['orderid', 'product', 'address'], 'sales_2019');
Using execute format() lets you replace all the quote_ident() with %I placeholders in a single text instead of a series of concatenated snippets. %1$I lets you re-use the first argument.
It's best if you use ARRAY['a','b','c']::VARCHAR(100)[] to explicitly make it an array of your desired type. '{"a","b","c"}'::VARCHAR(100)[] works too.
You'll need to convert the array into a list of columns some other way, because when cast to text, it'll get curly braces which are not allowed in the column list syntax. Demo
It's not a good practice to introduce random limitations - PostgreSQL doesn't limit identifier lengths to 100 characters, so you don't have to either. The default limit is 63 bytes, so you can go way, way longer than 100 characters (demo). You can switch that data type to a regular text. Interestingly, exceeding specified varchar length would just convert it to unlimited varchar, making it just syntax noise.
DBFiddle online demo
CREATE TABLE sales_2019(orderid INT,product INT,address INT);
CREATE OR REPLACE PROCEDURE data_versioning_nonull(
new_table_name TEXT,
column_names TEXT[],
current_table_name TEXT)
LANGUAGE plpgsql AS $$
DECLARE
list_of_columns_as_quoted_identifiers TEXT;
BEGIN
SELECT string_agg(quote_ident(name),',')
INTO list_of_columns_as_quoted_identifiers
FROM unnest(column_names) name;
EXECUTE format('CREATE TABLE %1$I.%2$I AS SELECT %3$s FROM %1$I.%4$I',
current_schema(),
new_table_name,
list_of_columns_as_quoted_identifiers,
current_table_name);
END $$;
CALL data_versioning_nonull(
'sales_2019_sample',
ARRAY['orderid', 'product', 'address']::text[],
'sales_2019');
Schema awareness: currently the procedure creates the new table in the default schema, based on a table in that same default schema - above I made it explicit, but that's what it would do without the current_schema() calls anyway. You could add new_table_schema and current_table_schema parameters and if most of the time you don't expect them to be used, you can hide them behind procedure overloads for convenience, using current_schema() to keep the implicit behaviour. Demo
First, change your stored procedure to convert selected columns from array to csv like this.
CREATE OR REPLACE PROCEDURE data_versioning_nonull(new_table_name VARCHAR(100),column_name VARCHAR(100)[], current_table_name VARCHAR(100))
language plpgsql
as $$
BEGIN
EXECUTE ('CREATE TABLE ' || quote_ident(new_table_name) || ' AS SELECT ' || array_to_string(column_name, ',') || ' FROM ' || quote_ident(current_table_name));
END $$;
Then call it as:
CALL data_versioning_nonull('sales_2019_sample', '{"orderid", "product", "address"}', 'sales_2019');
I am having a go at pgSQL for the first time.
Many thanks in advance indeed - Eugen
The problem I like to solve
I like to return an array of words with a minimum length from a text. Picked regexp_matches which returns SETOF text[] (not array). There may be better ways but this is what I picked.
regexp_matches ( string text, pattern text [, flags text ] ) → setof text[]
Postgres Reference for String functins
The Issue
When trying to use individual results from the resut set (seemingly arrays) they are in fact of type text but include the curly brackets of a Postgres array.
Example
Functionally the below example works because I added a workaround removing curly brackets with the trim function.
The output for the function described below
# select lower_words_arr('there is an answer','\w{3,}') AS words;
NOTICE: match = {there}
NOTICE: match = {answer}
words
----------------
{there,answer}
(1 row)
My Question
Why come, that individual regexp_matches entries match and return data as "{txt}" but behave as type text and not ARRAY?
Is there a way without my workaround?
Code
CREATE OR REPLACE FUNCTION lower_words_arr(input_str text, match_expr text) RETURNS text[] AS $$
DECLARE
words_arr text[];
one_match text;
BEGIN
FOR one_match IN SELECT regexp_matches(lower(input_str), match_expr, 'g')
AS match
LOOP
RAISE NOTICE 'match = %', one_match;
-- But if I write:
--
-- RAISE NOTICE 'match = %', one_match[0];
--
-- fails with:
---
-- ERROR: cannot subscript type text because it is not an array
-- CONTEXT: SQL statement "SELECT one_match[0]"
-- PL/pgSQL function lower_words_arr(text,text) line 10 at RAISE
--
-- even though this the result returned
---
-- NOTICE: match = {there}
-- NOTICE: match = {answewr}
words_arr := array_append(words_arr,trim(BOTH '{}' FROM one_match));
END LOOP;
RETURN words_arr;
END;
$$ LANGUAGE plpgsql;
I have now solved the issue without the trim workaround. The solution came right from the Postgres documentation. Look for SELECT (regexp_match('foobarbequebaz', 'bar.*que'))[1]; on that page. With that knowledge in hand I changed the function to
Improved Solution
CREATE OR REPLACE FUNCTION lower_words_arr(input_str text, match_expr text) RETURNS text[] AS $$
DECLARE
words_arr text[];
one_match text;
BEGIN
FOR one_match IN SELECT(regexp_matches(lower(input_str), match_expr, 'g'))[1] AS match
LOOP
RAISE NOTICE 'match = %', one_match;
words_arr := array_append(words_arr,one_match);
END LOOP;
RETURN words_arr;
END;
$$ LANGUAGE plpgsql STABLE;
Output
select lower_words_arr('there is an answer - you will find it in the Postgres documentation.','\w{4,}') AS words;
NOTICE: match = there
NOTICE: match = answer
NOTICE: match = will
NOTICE: match = find
NOTICE: match = postgres
NOTICE: match = documentation
words
-------------------------------------------------
{there,answer,will,find,postgres,documentation}
(1 row)
Conclusion
My original problem has been solved, but a puzzling piece understanding is still missing (not subject of my initial question):
Why does the one_match variable not behave like an array when [1] is removed from the SELECT?
Maybe I should open a question specific to that behaviour ...
I have a function that uses RECORD to temporarily store the data. I can use it - it's fine. My problem is that I can't hardcode columns I need to get from the RECORD. I must do it dynamically. Something line:
DECLARE
r1 RECORD;
r2 RECORD;
BEGIN
for r1 in Select column_name
from columns_to_process
where process_now = True
loop
for r2 in Select *
from my_data_table
where whatever
loop
-----------------------------
here I must call column by its name that is unknown at design time
-----------------------------
... do something with
r2.(r1.column_name)
end loop;
end loop;
END;
Does anyone know how to do it?
best regards
M
There is no need to select the all the qualifying rows and compute the total in a loop. Actually when working with SQL try to drop the word loop for your vocabulary; instead just use sum(column_name) in the select. The issue here is that you do not know what column to sum when the query is written, and all structural components(table names, columns names, operators, etc) must be known before submitting. You cannot use a variable for a structural component - in this case a column name. To do that you must use dynamic sql - i.e. SQL statement built by the process. The following accomplishes that: See example here.
create or replace function sum_something(
the_something text -- column name
, for_id my_table.id%type -- my_table.id
)
returns numeric
language plpgsql
as $$
declare
k_query_base constant text :=
$STMT$ Select sum(%I) from my_table where id = %s; $STMT$;
l_query text;
l_sum numeric;
begin
l_query = format(k_query_base, the_something, for_id);
raise notice E'Rumming Statememt:\n %',l_query; -- for prod raise Log
execute l_query into l_sum;
return l_sum;
end;
$$;
Well, after some time I figured out that I could use temporary table instead of RECORD. Doing so gives me all advantages of using dynamic queries so I can call any column by its name.
DECLARE
_my_var bigint;
BEGIN
create temporary table _my_temp_table as
Select _any, _column, _you, _need
from _my_table
where whatever = something;
execute 'Select ' || _any || ' from _my_temp_table' into _my_var;
... do whatever
END;
However I still believe that there should be a way to call records field by it's name.
I need to create a macro or function that takes in two parameters, and concats them together after a basic data manipulation. Then this text string should be able to be used in any other query.
CREATE OR REPLACE FUNCTION getData (_table text, _days text)
RETURNS text AS
$func$
SELECT $1 || to_char(CURRENT_TIMESTAMP - ($2 || ' days')::INTERVAL, 'YYYYMMDD');
$func$ LANGUAGE sql;
select * from getData('opportunity', '4') limit 10;
So what I am expecting from this is to actually get the same result as if I executed
select * from opportunity20151030 limit 10;
Instead I am getting "opportunity20151030"
EDIT:
The reason we need this is because my employer is doing a nightly snapshot of our salesforce data, about 17 objects in all. Thats why I can't return a table. Returning a table needs to specify the columns. But we need to be able to query a variety of tables. This really needs to be just a small utility macro. This way we can have one query, to generate graphs that compare data from today and a week ago. This can even be something inside of pgAdmin itself, and is not restricted to being postgresql function. Is there a way I can execute the function and use the result inline in another query. I spent an hour playing around with
Execute 'select * from $1' using getData('opportunity', '4')
type queries, but apparently the LANGUAGE specified changes what can and can't be used in terms of compatible SQL statements.
Thank You!
Well, your function is returning exactly what you asked for... the name of the table you want to look in.
Now, while you can do a function that returns some table's data, the function must know the "structure" of that data. Which means that you can't use it for any table but those that share the same structure (same number of fields, same data types in the same order). Guessing about your table's name i will say that is an inherited table and a function like the one belowe can do the trick for any derived (inherited) table from opportunity
CREATE OR REPLACE FUNCTION getData (_table text, _days text, _limit int default -1)
RETURNS SETOF opportunity AS
$func$
DECLARE
table_name text;
limit_clause text = ' ';
BEGIN
SELECT _table || to_char(CURRENT_TIMESTAMP - (_days || ' days')::INTERVAL, 'YYYYMMDD') INTO table_name;
IF _limit > -1 THEN
limit_clause = ' LIMIT ' || _limit;
END IF;
RETURN QUERY EXECUTE 'SELECT * FROM ' || table_name || limit_clause;
RETURN;
END
$func$ LANGUAGE plpgsql;
And use it like:
select * from getData('opportunity', '4', 10);
PS1: i put the limit as one extra parameter in the function, but because that parameter has a default value you can ignore it when calling the function and that will return all values. i put it there because otherwise the function will return all rows in the table and limit after that.
PS2: i would avoid doing this unless you really think is needed, because this cannot be optimized if put as part of a larger query.
I have a situation where I want to return the join between two views. and that's a lot of columns. It was pretty easy in sql server. But in PostgreSQL when I do the join. I get the error "a column definition list is required".
Is there any way I can bypass this, I don't want to provide the definitions of returning columns.
CREATE OR REPLACE FUNCTION functionA(username character varying DEFAULT ''::character varying, databaseobject character varying DEFAULT ''::character varying)
RETURNS SETOF ???? AS
$BODY$
Declare
SqlString varchar(4000) = '';
BEGIN
IF(UserName = '*') THEN
Begin
SqlString := 'select * from view1 left join ' + databaseobject + ' as view2 on view1.id = view2.id';
End;
ELSE
Begin
SqlString := 'select * from view3 left join ' + databaseobject + ' as view2 on view3.id = view2.id';
End;
END IF;
execute (SqlString );
END;
$BODY$
Sanitize function
What you currently have can be simplified / sanitized to:
CREATE OR REPLACE FUNCTION func_a (username text = '', databaseobject text = '')
RETURNS ????
LANGUAGE plpgsql AS
$func$
BEGIN
RETURN QUERY EXECUTE
format ('SELECT * FROM %s v1 LEFT JOIN %I v2 USING (id)'
, CASE WHEN username = '*' THEN 'view1' ELSE 'view3' END
, databaseobject);
END
$func$;
You only need additional instances of BEGIN ... END in the function body to start separate code blocks with their own scope, which is rarely needed.
The standard SQL concatenation operator is ||. + is a "creative" addition of your former vendor.
Don't use CaMeL-case identifiers unless you double-quote them. Best don't use them at all See:
Are PostgreSQL column names case-sensitive?
varchar(4000) is also tailored to a specific limitation of SQL Server. It has no specific significance in Postgres. Only use varchar(4000) if you actually need a limit of 4000 characters. I would just use text - except that we don't need any variables at all here, after simplifying the function.
If you have not used format(), yet, consult the manual here.
Return type
Now, for your actual question: The return type for a dynamic query can be tricky since SQL requires that to be declared at call time at the latest. If you have a table or view or composite type in your database already matching the column definition list, you can just use that:
CREATE FUNCTION foo()
RETURNS SETOF my_view AS
...
Else, spell the column definition list with out with (simplest) RETURNS TABLE:
CREATE FUNCTION foo()
RETURNS TABLE (col1 int, col2 text, ...) AS
...
If you are making the row type up as you go, you can return anonymous records:
CREATE FUNCTION foo()
RETURNS SETOF record AS
...
But then you have to provide a column definition list with every call, so I hardly ever use that.
I wouldn't use SELECT * to begin with. Use a definitive list of columns to return and declare your return type accordingly:
CREATE OR REPLACE FUNCTION func_a(username text = '', databaseobject text = '')
RETURNS TABLE(col1 int, col2 text, col3 date)
LANGUAGE plpgsql AS
$func$
BEGIN
RETURN QUERY EXECUTE
format ($f$SELECT v1.col1, v1.col2, v2.col3
FROM %s v1 LEFT JOIN %I v2 USING (id)$f$
, CASE WHEN username = '*' THEN 'view1' ELSE 'view3' END
, databaseobject);
END
$func$;
For completely dynamic queries, consider building the query in your client to begin with, instead of using a function.
You need to understand basics first:
Refactor a PL/pgSQL function to return the output of various SELECT queries
PL/pgSQL in the Postgres manual
Then there are more advanced options with polymorphic types, which allow you to pass the return type at call time. More in the last chapter of:
Refactor a PL/pgSQL function to return the output of various SELECT queries