Postgresql9-4 - User define function that uses a join - ERROR: missing FROM-clause entry for table - plpgsql

I am venturing into pg/sql for the first time and I wrote this function that (attempts) to return results from a query that uses a JOIN.
The query works when I run on it self. However, when it is running within the function I wrote it complains about a missing "FROM". I also used table aliases hoping that it may fix the problem but it did not.
I am getting information from the following tables and rows.
TABLE | ROW
---------------------------------------
banned_users | banned_lcl_account
---------------------------------------
rhost_active_users | active_users
Here is the error when I run my function
ERROR: missing FROM-clause entry for table "a_u"
LINE 1: SELECT LOOP_V.a_u.ipaddress
And here is the function I wrote.
CREATE OR REPLACE FUNCTION GET_BANNED_ACTIVE_USERS()
RETURNS TABLE
(
SRC_HOST TEXT,
DST_HOST TEXT
)
AS $$
DECLARE
LOOP_V RECORD;
BEGIN
FOR LOOP_V IN (
select
b_u.banned_lcl_account,
a_u.ipaddress,
a_u.hostip,
date_trunc ('second', a_u.time_captured)
from banned_users as b_u
inner join rhost_active_users as a_u
on b_u.banned_lcl_account = a_u.active_users
and a_u.hostip <> 'TTY Login'
or a_u.hostip <> 'Local PTS Login'
)
LOOP
SRC_HOST := LOOP_V.a_u.ipaddress;
DST_HOST := LOOP_V.a_u.hostip;
END LOOP;
END; $$
LANGUAGE 'plpgsql';
--select CHECK_BANNED_ACCOUNTS()

your problem is this:
SRC_HOST := LOOP_V.a_u.ipaddress;
DST_HOST := LOOP_V.a_u.hostip;
The record that is used for the loop variable only contains column names, not their table aliases. So the record only has the fields: ipaddress and hostip. So your immediate problem can be fixed using:
SRC_HOST := LOOP_V.ipaddress;
DST_HOST := LOOP_V.hostip;
However the whole function is overly complex. You don't need the loop nor do you need PL/pgSQL. This can be written as a simple SQL function:
CREATE OR REPLACE FUNCTION GET_BANNED_ACTIVE_USERS()
RETURNS TABLE (SRC_HOST TEXT,DST_HOST TEXT)
AS $$
select a_u.ipaddress,
a_u.hostip,
from banned_users as b_u
join rhost_active_users as a_u
on b_u.banned_lcl_account = a_u.active_users
and (a_u.hostip <> 'TTY Login' or a_u.hostip <> 'Local PTS Login')
$$
LANGUAGE sql;
Note that I also removed the unused columns from the SELECT statement in order to match the function's signature of returning two columns
Some additional notes:
--select CHECK_BANNED_ACCOUNTS()
This is wrong, you need to use a set returning function like a table:
select *
from CHECK_BANNED_ACCOUNTS();
And finally:
LANGUAGE 'plpgsql';
The language name is an identifier, do not enclose it in single quotes:
LANGUAGE plpgsql;
or
LANGUAGE sql;

Related

Call postgresql record's field by name

I have a function that uses RECORD to temporarily store the data. I can use it - it's fine. My problem is that I can't hardcode columns I need to get from the RECORD. I must do it dynamically. Something line:
DECLARE
r1 RECORD;
r2 RECORD;
BEGIN
for r1 in Select column_name
from columns_to_process
where process_now = True
loop
for r2 in Select *
from my_data_table
where whatever
loop
-----------------------------
here I must call column by its name that is unknown at design time
-----------------------------
... do something with
r2.(r1.column_name)
end loop;
end loop;
END;
Does anyone know how to do it?
best regards
M
There is no need to select the all the qualifying rows and compute the total in a loop. Actually when working with SQL try to drop the word loop for your vocabulary; instead just use sum(column_name) in the select. The issue here is that you do not know what column to sum when the query is written, and all structural components(table names, columns names, operators, etc) must be known before submitting. You cannot use a variable for a structural component - in this case a column name. To do that you must use dynamic sql - i.e. SQL statement built by the process. The following accomplishes that: See example here.
create or replace function sum_something(
the_something text -- column name
, for_id my_table.id%type -- my_table.id
)
returns numeric
language plpgsql
as $$
declare
k_query_base constant text :=
$STMT$ Select sum(%I) from my_table where id = %s; $STMT$;
l_query text;
l_sum numeric;
begin
l_query = format(k_query_base, the_something, for_id);
raise notice E'Rumming Statememt:\n %',l_query; -- for prod raise Log
execute l_query into l_sum;
return l_sum;
end;
$$;
Well, after some time I figured out that I could use temporary table instead of RECORD. Doing so gives me all advantages of using dynamic queries so I can call any column by its name.
DECLARE
_my_var bigint;
BEGIN
create temporary table _my_temp_table as
Select _any, _column, _you, _need
from _my_table
where whatever = something;
execute 'Select ' || _any || ' from _my_temp_table' into _my_var;
... do whatever
END;
However I still believe that there should be a way to call records field by it's name.

Dynamic columns in SQL statement

I am trying to have a dynamic variable that I can specify different column's with (depending on some if statements). Explained in code, I am trying to replace this:
IF (TG_TABLE_NAME='this') THEN INSERT INTO table1 (name_id) VALUES id.NEW END IF;
IF (TG_TABLE_NAME='that') THEN INSERT INTO table1 (lastname_id) VALUES id.NEW END IF;
IF (TG_TABLE_NAME='another') THEN INSERT INTO table1 (age_id) VALUES id.NEW END IF;
With this:
DECLARE
varName COLUMN;
BEGIN
IF (TG_TABLE_NAME='this') THEN varName = 'name_id';
ELSE IF (TG_TABLE_NAME='that') THEN varName = 'lastname_id';
ELSE (TG_TABLE_NAME='another') THEN varName = 'age_id';
END IF;
INSERT INTO table1 (varName) VALUES id.NEW;
END;
The INSERT string is just an example, it's actually something longer. I am a beginner at pgSQL. I've seen some examples but I'm only getting more confused. If you can provide an answer that is also more safe from SQL injection that would be awesome.
One way to do what you're looking for is to compose your INSERT statement dynamically based on the named table. The following function approximates the logic you laid out in the question:
CREATE OR REPLACE FUNCTION smart_insert(table_name TEXT) RETURNS VOID AS $$
DECLARE
target TEXT;
statement TEXT;
BEGIN
CASE table_name
WHEN 'this' THEN target := 'name_id';
WHEN 'that' THEN target := 'lastname_id';
WHEN 'another' THEN target := 'age_id';
END CASE;
statement :=
'INSERT INTO '||table_name||'('||target||') VALUES (nextval(''id''));';
EXECUTE statement;
END;
$$ LANGUAGE plpgsql;
Note that I'm using a sequence to populate these tables (the call to nextval). I'm not sure if that is your use case, but hopefully this example is extensible enough for you to modify it to fit your scenario. A contrived demo:
postgres=# SELECT smart_insert('this');
smart_insert
--------------
(1 row)
postgres=# SELECT smart_insert('that');
smart_insert
--------------
(1 row)
postgres=# SELECT name_id FROM this;
name_id
---------
101
(1 row)
postgres=# SELECT lastname_id FROM that;
lastname_id
-------------
102
(1 row)
Your example doesn't make a lot of sense. Probably over-simplified. Anyway, here is a trigger function for the requested functionality that inserts the new id in a selected column of a target table, depending on the triggering table:
CREATE OR REPLACE FUNCTION smart_insert(table_name TEXT)
RETURNS trigger AS
$func$
BEGIN
EXECUTE
'INSERT INTO table1 ('
|| CASE TG_TABLE_NAME
WHEN 'this' THEN 'name_id'
WHEN 'that' THEN 'lastname_id'
WHEN 'another' THEN 'age_id'
END CASE
||') VALUES ($1)'
USING NEW.id;
END
$func$ LANGUAGE plpgsql;
To refer to the id column of the new row, use NEW.id not id.NEW.
To pass a value to dynamic code, use the USING clause of EXECUTE. This is faster and more elegant, avoids casting to text and back and also makes SQL injection impossible.
Don't use many variables and assignments in plpgsql, where this is comparatively expensive.
If the listed columns of the target table don't have non-default column defaults, you don't even need dynamic SQL:
CREATE OR REPLACE FUNCTION smart_insert(table_name TEXT)
RETURNS trigger AS
$func$
BEGIN
INSERT INTO table1 (name_id, lastname_id, age_id)
SELECT CASE WHEN TG_TABLE_NAME = 'this' THEN NEW.id END
, CASE WHEN TG_TABLE_NAME = 'that' THEN NEW.id END
, CASE WHEN TG_TABLE_NAME = 'another' THEN NEW.id END;
END
$func$ LANGUAGE plpgsql;
A CASE expression without ELSE clause defaults to NULL, which is the default column default.
Both variants are safe against SQL injection.

How to return multiple rows from PL/pgSQL function?

I have spent good amount of time trying to figure it out and I haven't been able to resolve it. So, I need your help please.
I am trying to write a PL/pgSQL function that returns multiple rows. The function I wrote is shown below. But it is not working.
CREATE OR REPLACE FUNCTION get_object_fields()
RETURNS SETOF RECORD
AS
$$
DECLARE result_record keyMetrics;
BEGIN
return QUERY SELECT department_id into result_record.visits
from fact_department_daily
where report_date='2013-06-07';
--return result_record;
END
$$ LANGUAGE plpgsql;
SELECT * FROM get_object_fields;
It is returning this error:
ERROR: RETURN cannot have a parameter in function returning set;
use RETURN NEXT at or near "QUERY"
After fixing the bugs #Pavel pointed out, also define your return type properly, or you have to provide a column definition list with every call.
This call:
SELECT * FROM get_object_fields()
... assumes that Postgres knows how to expand *. Since you are returning anonymous records, you get an exception:
ERROR: a column definition list is required for functions returning "record"
One way (of several) to fix this is with RETURNS TABLE (Postgres 8.4+):
CREATE OR REPLACE FUNCTION get_object_fields()
RETURNS TABLE (department_id int) AS
$func$
BEGIN
RETURN QUERY
SELECT department_id
FROM fact_department_daily
WHERE report_date = '2013-06-07';
END
$func$ LANGUAGE plpgsql;
Works for SQL functions just the same.
Related:
PostgreSQL: ERROR: 42601: a column definition list is required for functions returning "record"
I see more bugs:
first, a SET RETURNING FUNCTIONS call has following syntax
SELECT * FROM get_object_fields()
second - RETURN QUERY forwards query result to output directly. You cannot store this result to variable - it is not possible ever in PostgreSQL now.
BEGIN
RETURN QUERY SELECT ....; -- result is forwarded to output directly
RETURN; -- there will not be any next result, finish execution
END;
third - these simple functions is better to implement in SQL languages
CREATE OR REPLACE FUNCTION get_object_fields()
RETURNS SETOF RECORD AS $$
SELECT department_id WHERE ...
$$ LANGUAGE sql STABLE;
Here's one way
drop function if exists get_test_type();
drop type if exists test_comp;
drop type if exists test_type;
drop type if exists test_person;
create type test_type as (
foo int,
bar int
);
create type test_person as (
first_name text,
last_name text
);
create type test_comp as
(
prop_a test_type[],
prop_b test_person[]
);
create or replace function get_test_type()
returns test_comp
as $$
declare
a test_type[];
b test_person[];
x test_comp;
begin
a := array(
select row (m.message_id, m.message_id)
from message m
);
-- alternative 'strongly typed'
b := array[
row('Bob', 'Jones')::test_person,
row('Mike', 'Reid')::test_person
]::test_person[];
-- alternative 'loosely typed'
b := array[
row('Bob', 'Jones'),
row('Mike', 'Reid')
];
-- using a select
b := array (
select row ('Jake', 'Scott')
union all
select row ('Suraksha', 'Setty')
);
x := row(a, b);
return x;
end;
$$
language 'plpgsql' stable;
select * from get_test_type();
CREATE OR REPLACE FUNCTION get_object_fields()
RETURNS table (department_id integer)
AS
$$
DECLARE result_record keyMetrics;
BEGIN
return QUERY
SELECT department_id
from fact_department_daily
where report_date='2013-06-07';
--return result_record;
END;
$$ LANGUAGE plpgsql;
SELECT * FROM get_object_fields()

Using prepared statement in stored function

I have a table in the database:
create table store (
...
n_status integer not null,
t_tag varchar(4)
t_name varchar,
t_description varchar,
dt_modified timestamp not null,
...
);
In my stored function I need to execute the same select against this table multiple times:
select * from store
where n_place_id = [different values]
and t_tag is not null
and n_status > 0
and (t_name ~* t_search or t_description ~* t_search)
order by dt_modified desc
limit n_max;
Here, t_search and n_max are parameters into the stored function. I thought it would make sense to use a prepared statement for this, but I'm running into strange problems. Here's what I have:
create or replace function fn_get_data(t_search varchar, n_max integer)
returns setof store as
$body$
declare
resulter store%rowtype;
mid integer;
begin
prepare statement prep_stmt(integer) as
select *
from store
where n_place_id = $1
and (t_name ~* t_search or t_description ~* t_search)
order by dt_modified
limit n_max;
for mid in
(select n_place_id from ... where ...)
loop
for resulter in
execute prep_stmt(mid)
loop
return next resulter;
end loop;
end loop;
end;$body$
language 'plpgsql' volatile;
However when I actually run the function with
select * from fn_get_data('', 30)
I receive this error:
ERROR: column "t_search" does not exist
LINE 3: and (t_name ~* t_search or t_description ~* t_search)
^
QUERY: prepare prep_stmt(integer) as
select * from store where n_status > 0 and t_tag is not null and n_museum = $1
and (t_name ~* t_search or t_description ~* t_search)
order by dt_modified desc limit maxres_free
Ok, maybe it doesn't like external variables in the prepared statement, so I changed this to be
prepare prep_stmt(integer, varchar, integer) as
select * from store where n_status > 0 and t_tag is not null and n_museum = $1
and (t_name ~* $2 or t_description ~* $2)
order by dt_modified desc limit $3
...
for resulter in
execute prep_stmt(mid, t_search, n_max)
...
This time I get a different error:
ERROR: function prep_stmt(integer, character varying, integer) does not exist
LINE 1: SELECT prep_stmt(mid, t_search, n_max)
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
QUERY: SELECT prep_stmt(mid, t_search, n_max)
What am I missing here?
EDIT I added the relevant table structure at the top.
Looks to me like the PL/PgSQL EXECUTE for dynamic SQL trumps the regular SQL EXECUTE for prepared statements.
Code:
create or replace function prep_test() returns void as $$
begin
PREPARE do_something AS SELECT 1;
EXECUTE do_something;
end;
$$ LANGUAGE 'plpgsql';
Test:
regress=# select prep_test(1);
ERROR: column "do_something" does not exist
LINE 1: SELECT do_something
^
QUERY: SELECT do_something
CONTEXT: PL/pgSQL function "prep_test" line 4 at EXECUTE statement
outside PL/PgSQL it works fine:
regress=# EXECUTE do_something;
?column?
----------
1
(1 row)
I'm not sure how you'd execute a prepared statement within PL/PgSQL.
Out of interest, why are you trying to use prepared statements within PL/PgSQL? Plans are prepared and cached for PL/PgSQL anyway, it happens automatically.
There is a way to EXECUTE a prepared statement in a function, but like the accepted answer said, you typically don't wan't to do this in a function because the function already stores its plan.
That being said, there are still use cases where you do need to use a prepared statement in a function. My use case for this is when using multiple schemas for different users where the schemas contain tables that are similarly named and you want to use the same function to access one of these tables based off of what the search_path is set to. In this situation, because of the way the function stores its plan, using the same function after changing the search_path causes things to break. There are two solutions to this problem that I've stated. The first is to use EXECUTE '<Your query as a string here>'. But this can get very ugly for large queries, hence the reason to use the second method, which involves a PREPARE.
So with the background as to 'why' you'd want to do this out of the way, here is the how:
CREATE OR REPLACE FUNCTION prep_test()
RETURNS void AS $$
BEGIN
PREPARE do_something AS SELECT 1;
EXECUTE 'EXECUTE do_something;';
END;
$$ LANGUAGE plpgsql;
Though it will probably be in your best interests to add some protections to keep it from breaking. Something like:
CREATE OR REPLACE FUNCTION prep_test()
RETURNS void AS $$
BEGIN
IF (SELECT count(*) FROM pg_prepared_statements WHERE name ilike 'do_something') > 0 THEN
DEALLOCATE do_something;
END IF;
PREPARE do_something AS SELECT 1;
EXECUTE 'EXECUTE do_something;';
DEALLOCATE do_something;
END;
$$ LANGUAGE plpgsql;
Again, those who think that they want to do this, usually probably shouldn't, but for those cases where it is needed, this is how you do it.
You could use an EXECUTE statement like this in PLPGSQL:
select magicvalue into str_execute from magicvalues where magickey = ar_requestData[2];
EXECUTE str_execute into str_label USING ar_requestData[3], ar_requestData[4]::boolean, ar_requestData[5]::int, ar_requestData[6];
This is code I use in my application. ar_requestData is an array with text values.
In the table magicvalues do I store things like prepared statements.
The select statement is for example:
insert into classtypes(label, usenow, ranking, description) values($1,$2,$3,$4) returning label'
With kind regards,
Loek Bergman
PREPARE statement is not allowed within plpgsql. You can splice all the statements inside a function and use dynamic execute after then. Here is an example.
create or replace function sp_test(f_total int) returns void as $ytt$
declare v_sql text;
declare i int;
begin
v_sql:='prepare ytt_s1 (int,timestamp) as select * from tbl1 where id = $1 and log_time = $2;';
while i < f_total
loop
v_sql:=v_sql||'execute ytt_s1('||i||',now());';
i := i + 1;
end loop;
v_sql:=v_sql||'deallocate ytt_s1;';
execute v_sql;
end;
$ytt$ language plpgsql;

I want to have my pl/pgsql script output to the screen

I have the following script that I want output to the screen from.
CREATE OR REPLACE FUNCTION randomnametest() RETURNS integer AS $$
DECLARE
rec RECORD;
BEGIN
FOR rec IN SELECT * FROM my_table LOOP
SELECT levenshtein('mystring',lower('rec.Name')) ORDER BY levenshtein;
END LOOP;
RETURN 1;
END;
$$ LANGUAGE plpgsql;
I want to get the output of the levenshein() function in a table along with the rec.Name. How would I do that? Also, it is giving me an error about the line where I call levenshtein(), saying that I should use perform instead.
Assuming that you want to insert the function's return value and the rec.name into a different table. Here is what you can do (create the table new_tab first)-
SELECT levenshtein('mystring',lower(rec.Name)) AS L_val;
INSERT INTO new_tab (L_val, rec.name);
The usage above is demonstrated below.
I guess, you can use RAISE INFO 'This is %', rec.name; to view the values.
CREATE OR REPLACE FUNCTION randomnametest() RETURNS integer AS $$
DECLARE
rec RECORD;
BEGIN
FOR rec IN SELECT * FROM my_table LOOP
SELECT levenshtein('mystring',lower(rec.Name))
AS L_val;
RAISE INFO '% - %', L_val, rec.name;
END LOOP;
RETURN 1;
END;
$$ LANGUAGE plpgsql;
Note- the FROM clause is optional in case you select from a function in a select like netxval(sequence_name) and don't have any actual table to select from i.e. like SELECT nextval(sequence_name) AS next_value;, in Oracle terms it would be SELECT sequence_name.nextval FROM dual; or SELECT function() FROM dual;. There is no dual in postgreSQL.
I also think that the ORDER BY is not necessary since my assumption would be that your function levenshtein() will most likely return only one value at any point of time, and hence wouldn't have enough data to ORDER.
If you want the output from a plpgsql function like the title says:
CREATE OR REPLACE FUNCTION randomnametest(_mystring text)
RETURNS TABLE (l_dist int, name text) AS
$BODY$
BEGIN
RETURN QUERY
SELECT levenshtein(_mystring, lower(t.name)), t.name
FROM my_table t
ORDER BY 1;
END;
$$ LANGUAGE plpgsql;
Declare the table with RETURNS TABLE.
Use RETURN QUERY to return records from the function.
Avoid naming conflicts between column names and OUT parameters (from the RETURNS TABLE clause) by table-qualifying column names in queries. OUT parameters are visible everywhere in the function body.
I made the string to compare to a parameter to the function to make this more useful.
There are other ways, but this is the most effective for the task. You need PostgreSQL 8.4 or later.
For a one-time use I would consider to just use a plain query (= function body without the RETURN QUERY above).