Is there a concise way to select the nextval for a PostgreSQL sequence multiple times in 1 query? This would be the only value being returned.
For example, I would like to do something really short and sweet like:
SELECT NEXTVAL('mytable_seq', 3) AS id;
And get:
id
-----
118
119
120
(3 rows)
select nextval('mytable_seq') from generate_series(1,3);
generate_series is a function which returns many rows with sequential numbers, configured by it's arguments.
In above example, we don't care about the value in each row, we just use generate_series as row generator. And for each row we can call nextval. In this case it returns 3 numbers (nextvals).
You can wrap this into function, but I'm not sure if it's really sensible given how short the query is.
There is a great article about this exact problem: "getting multiple values from sequences".
If performance is not an issue, for instance when using the sequence values dwarfs the time used to get them or n is small, then the SELECT nextval('seq') FROM generate_series(1,n) approach is the simplest and most appropriate.
But when preparing data for bulk loads the last approach from the article of incrementing the sequence by n from within a lock is appropriate.
CREATE OR REPLACE FUNCTION foo() RETURNS SETOF INT AS $$
DECLARE
seqval int; x int;
BEGIN
x := 0;
WHILE x < 100 LOOP
SELECT into seqval nextval('f_id_seq');
RETURN NEXT seqval;
x := x+1;
END LOOP;
RETURN;
END;
$$ LANGUAGE plpgsql STRICT;
Of course, if all you're trying to do is advance the sequence, there's setval().
You could also have the function take a parameter for how many times to loop:
CREATE OR REPLACE FUNCTION foo(loopcnt int) RETURNS SETOF INT AS $$
DECLARE
seqval int;
x int;
BEGIN
x := 0;
WHILE x < loopcnt LOOP
SELECT into seqval nextval('f_id_seq');
RETURN NEXT seqval;x := x+1;
END LOOP;
RETURN;
END;
$$ LANGUAGE plpgsql STRICT;
Unless you really want three rows returned I would set the sequence to 'INCREMENT BY 3' for each select. Then you can simple add 1 and 2 to the result have have your three sequence numbers.
I tried to add a link to the postgresql docs, but apparenty I am not allowed to post links.
My current best solution is:
SELECT NEXTVAL('mytable_seq') AS id
UNION ALL
SELECT NEXTVAL('mytable_seq') AS id
UNION ALL
SELECT NEXTVAL('mytable_seq') AS id;
Which will correctly return 3 rows... but I would like something that is minimal SQL for even as much as 100 or more NEXTVAL invocations.
Related
Suppose I have:
CREATE TYPE compfoo AS (f1 int, f2 text);
And I create a table foo containing two columns: fooid and fooname, corresponding to the fields of compfoo, later I insert some records 1, aa, 2, bb, 3, cc
Then, I define a PL/pgSQL function (more or less as follows:)
create or replace function foo_query()
returns text
language plpgsql
as $$
declare
r compfoo;
arr compfoo [];
footemp compfoo;
result text;
begin
for r in
select * from foo where fooid = 1 OR fooid = 2
loop
arr := array_append(arr, r);
end loop;
foreach footemp in array arr
loop
select footemp.f1 into result where footemp.f1 = 1;
end loop;
return result;
end;
$$
Where I query first foo by column names and save the results into arr, an array of compfoo. Later, I iterate over arr and try to query the elements by their fieldnames as defined in compfoo.
I don't get an error per se in Postgres but the result of my function is null.
What am I doing wrong?
The RAISE NOTICE should be your best friend. You can print the result of some variables at some points of your code. The basic issue are not well initialized values. The arr variable is initialized by NULL value, and any operation over NULL is NULL again.
Another problem is in select footemp.f1 into result where footemp.f1 = 1; statement. SELECT INTO in Postgres overwrite the target variable by NULL value when an result is empty. In second iteration, the result of this query is empty set, and the result variable is set on NULL.
The most big problem of your example is style of programming. You use ISAM style, and your code can be terrible slow.
Don't use array_append in cycle, when you can use array_agg function in query, and you don't need cycle,
Don't use SELECT INTO when you don't read data from tables,
Don't try to repeat Oracle' pattern BULK COLLECT and FOREACH read over collection. PostgreSQL is not Oracle, uses very different architecture, and this pattern doesn't increase performance (like on Oracle), but probably you will lost some performance.
Your fixed code can looks like:
CREATE OR REPLACE FUNCTION public.foo_query()
RETURNS text
LANGUAGE plpgsql
AS $function$
declare
r compfoo;
arr compfoo [] default '{}'; --<<<
footemp compfoo;
result text;
begin
for r in
select * from foo where fooid = 1 or fooid = 2
loop
arr := array_append(arr, r);
end loop;
foreach footemp in array arr
loop
if footemp.f1 = 1 then --<<<
result := footemp.f1;
end if;
end loop;
return result;
end;
$function$
postgres-# ;
It returns expected result. But it is perfect example how don't write stored procedures. Don't try to replace SQL in stored procedures. All code of this procedure can be replaced just by one query. In the end this code can be very slow on bigger data.
I have a function that uses RECORD to temporarily store the data. I can use it - it's fine. My problem is that I can't hardcode columns I need to get from the RECORD. I must do it dynamically. Something line:
DECLARE
r1 RECORD;
r2 RECORD;
BEGIN
for r1 in Select column_name
from columns_to_process
where process_now = True
loop
for r2 in Select *
from my_data_table
where whatever
loop
-----------------------------
here I must call column by its name that is unknown at design time
-----------------------------
... do something with
r2.(r1.column_name)
end loop;
end loop;
END;
Does anyone know how to do it?
best regards
M
There is no need to select the all the qualifying rows and compute the total in a loop. Actually when working with SQL try to drop the word loop for your vocabulary; instead just use sum(column_name) in the select. The issue here is that you do not know what column to sum when the query is written, and all structural components(table names, columns names, operators, etc) must be known before submitting. You cannot use a variable for a structural component - in this case a column name. To do that you must use dynamic sql - i.e. SQL statement built by the process. The following accomplishes that: See example here.
create or replace function sum_something(
the_something text -- column name
, for_id my_table.id%type -- my_table.id
)
returns numeric
language plpgsql
as $$
declare
k_query_base constant text :=
$STMT$ Select sum(%I) from my_table where id = %s; $STMT$;
l_query text;
l_sum numeric;
begin
l_query = format(k_query_base, the_something, for_id);
raise notice E'Rumming Statememt:\n %',l_query; -- for prod raise Log
execute l_query into l_sum;
return l_sum;
end;
$$;
Well, after some time I figured out that I could use temporary table instead of RECORD. Doing so gives me all advantages of using dynamic queries so I can call any column by its name.
DECLARE
_my_var bigint;
BEGIN
create temporary table _my_temp_table as
Select _any, _column, _you, _need
from _my_table
where whatever = something;
execute 'Select ' || _any || ' from _my_temp_table' into _my_var;
... do whatever
END;
However I still believe that there should be a way to call records field by it's name.
I have a long and complex plpgsql function that creates a bunch of temporary tables nested within a while statement to get the optimal result. When the condition has been met I insert the result into an existing table, the function is far to long to post here but this is an example:
CREATE OR REPLACE FUNCTION public.test_function(id_input integer, val_input numeric)
RETURNS VOID AS
$BODY$
DECLARE
id_input numeric = $1;
val_input numeric = $2;
BEGIN
WHILE test_val < 0
LOOP
CREATE TEMP TABLE temp_table AS
SELECT a.existing_val - val_input AS new_val
FROM existing_table a
WHERE a.id = id_input;
test_val := (SELECT new_val FROM temp_table);
val_input := val_input + 1;
END LOOP;
INSERT INTO output_table (id, new_val)
SELECT a.id, a.new_val
FROM temp_table a;
END;
$BODY$
LANGUAGE plpgsql;
The function works if I call it like this SELECT test_function(1, 1000) However I would like run this function on a table with 60,000+ rows, like this:
SELECT test_function(a.id, a.val_input)
FROM data_table a;
It works when I use a subset of the data_table, say 1000 rows. However when I run it on the full table (60,000+ rows) I get the following error "AbortTransaction while in COMMIT state". After some reading I found out COMMITS, so in my case the inserts do not occur until the function has finished running which takes about 4 hours. So does anyone know what is going on?
As a workaround I tried nesting the function in a DO statement so the inserts are committed straight away:
DO
$do$
DECLARE
r data_table%rowtype;
BEGIN
FOR r IN
SELECT * FROM data_table
LOOP
SELECT public.test_function(r.id, r.val_input);
END LOOP;
END
$do$;
However then I get the following error "ERROR: query has no destination for result data", which I guess means I need to rewrite the function to use PERFORM instead of SELECT. However I have not had any luck with this as yet.
Any ideas?
Since you are not interested in the function result, you should use
PERFORM public.test_function(r.id, r.val_input);
instead of
SELECT public.test_function(r.id, r.val_input);
The latter syntax would only work if you add INTO some_variable as a destination for the query result.
Thank you all for your suggestions. I ended up using Jim Jones's suggestion and converting the function to a procedure which allowed me to use COMMIT after I did the INSERT. I also followed Jeremy's suggestion and moved from temp tables to CTE's. This solved the problem for me.
I have a Postgres function which is returning a table:
CREATE OR REPLACE FUNCTION testFunction() RETURNS TABLE(a int, b int) AS
$BODY$
DECLARE a int DEFAULT 0;
DECLARE b int DEFAULT 0;
BEGIN
CREATE TABLE tempTable AS SELECT a, b;
RETURN QUERY SELECT * FROM tempTable;
DROP TABLE tempTable;
END;
$BODY$
LANGUAGE plpgsql;
This function is not returning data in row and column form. Instead it returns data as:
(0,0)
That is causing a problem in Coldfusion cfquery block in extracting data. How do I get data in rows and columns when a table is returned from this function? In other words: Why does the PL/pgSQL function not return data as columns?
To get individual columns instead of the row type, call the function with:
SELECT * FROM testfunction();
Just like you would select all columns from a table.
Also consider this reviewed form of your test function:
CREATE OR REPLACE FUNCTION testfunction()
RETURNS TABLE(a int, b int)
LANGUAGE plpgsql AS
$func$
DECLARE
_a int := 0;
_b int := 0;
BEGIN
CREATE TABLE tempTable AS SELECT _a, _b;
RETURN QUERY SELECT * FROM tempTable;
DROP TABLE tempTable;
END
$func$;
In particular:
The DECLARE key word is only needed once.
Avoid declaring parameters that are already (implicitly) declared as OUT parameters in the RETURNS TABLE (...) clause.
Don't use unquoted CaMeL-case identifiers in Postgres. It works, unquoted identifiers are cast to lower case, but it can lead to confusing errors. See:
Are PostgreSQL column names case-sensitive?
The temporary table in the example is completely useless (probably over-simplified). The example as given boils down to:
CREATE OR REPLACE FUNCTION testfunction(OUT a int, OUT b int)
LANGUAGE plpgsql AS
$func$
BEGIN
a := 0;
b := 0;
END
$func$;
Of course you can do this by putting the function call in the FROM clause, like Eric Brandstetter correctly answered.
However, this is sometimes complicating in a query that already has other things in the FROM clause.
To get the individual columns that the function returns, you can use this syntax:
SELECT (testfunction()).*
Or to get only the column called "a":
SELECT (testfunction()).a
Place the whole function, including the input value(s) in parenteses, followed by a dot and the desired column name, or an asterisk.
To get the column names that the function returns, you'll have to either:
check the source code
inspect the result of the function first, like so : SELECT * FROM testfunction() .
The input values can still come out of a FROM clause.
Just to illustrate this, consider this function and test data:
CREATE FUNCTION funky(a integer, b integer)
RETURNS TABLE(x double precision, y double precision) AS $$
SELECT a*random(), b*random();
$$ LANGUAGE SQL;
CREATE TABLE mytable(a integer, b integer);
INSERT INTO mytable
SELECT generate_series(1,100), generate_series(101,200);
You could call the function "funky(a,b)", without the need to put it in the FROM clause:
SELECT (funky(mytable.a, mytable.b)).*
FROM mytable;
Which would result in 2 columns:
x | y
-------------------+-------------------
0.202419687062502 | 55.417385618668
1.97231830470264 | 63.3628275180236
1.89781916560605 | 1.98870931006968
(...)
I'm trying to fasten posgresql selection using a function by means of saying it's immutable or stable, so I have a function
CREATE OR REPLACE FUNCTION get_data(uid uuid)
RETURNS integer AS $$
BEGIN
RAISE NOTICE 'UUID %', $1;
-- DO SOME STUFF
RETURN 0;
END;
$$ LANGUAGE plpgsql IMMUTABLE STRICT;
When I call it like:
SELECT get_data('3642e529-b098-4db4-b7e7-6bb62f8dcbba'::uuid)
FROM table
WHERE true LIMIT 100;
I have 100 results and only one notice raised
When I call it this way:
SELECT get_data(table.hash)
FROM table
WHERE 1 = 1 AND table.hash = '3642e529-b098-4db4-b7e7-6bb62f8dcbba' LIMIT 100;
I have 100 result and 100 notices raised
the condition (table.hash = '3642e529-b098-4db4-b7e7-6bb62f8dcbba') added to make sure that the in param is the same
table.hash is uuid type
The questions is:
So how can force PG to some how cache the result of the function? ( if it's possibe )
I want to have only one notice ( function call ) be raised in the second case...
In your first example get_data('3642e529-b098-4db4-b7e7-6bb62f8dcbba'::uuid) is a constant, independent of table rows, so it is evaluated once.
In the second example get_data(table.hash) is functionally depending on a column value, therefore it is evaluated once per row.
If you want to evaluate the function once, it cannot depend on a value from a column (when more than one row is processed).
After discussion in comments, here is an example how to call function only once per hash:
SELECT *, get_data(x.hash) AS some_data_once_per_hash
FROM (
SELECT hash, count(*) AS ct
FROM table
WHERE table.hash = '3642e529-b098-4db4-b7e7-6bb62f8dcbba'
GROUP BY 1
) x
If Erwin's answer is not good for your case you can either create a materialized view or a trigger to update a "computed column"