INSERT while LOOPing - postgresql

i'd like to insert some records into a table while looping. Records are passed as a varchar tab to a function ie.:
create function a(tab varchar[]) RETURNS void AS
$$
DECLARE
b varchar(20);
BEGIN
FOREACH b IN ARRAY tab
LOOP
INSERT INTO....;
END LOOP;
RETURN;
END;
$$
LANGUAGE plpgsql;
However, when executing there is the error:
ERROR: syntax error at or near "FOREACH"
LINE 1: FOREACH $1 IN ARRAY tab LOOP INSERT INTO x (c, ...
QUERY: FOREACH $1 IN ARRAY tab LOOP INSERT INTO x (c) VALUES ( $1)
CONTEXT: SQL statement in PL/PgSQL function " near line ...
********** ERROR **********
ERROR: syntax error at or near "FOREACH"
Does anybody know why and how to fix this?

FOREACH construct was added in 9.1. For earlier versions you can use unnest() function the following way:
CREATE FUNCTION a(tab varchar[]) RETURNS void AS
$$
DECLARE
b varchar(20);
BEGIN
FOR b IN SELECT unnest(tab)
LOOP
INSERT INTO x(c) VALUES (b);
END LOOP;
RETURN;
END;
$$ LANGUAGE plpgsql;

Since PostgreSQL 9.1 there is also a built-in way to loop through array slices:
FOREACH x SLICE 1 IN ARRAY $1
LOOP
RAISE NOTICE 'row = %', x;
END LOOP;
Where x must be a matching array type and ...
The SLICE value must be an integer constant not larger than the number of dimensions of the array
For 1-dimensional arrays just omit the SLICE part and x can be a simple type like you display in your question.

I want to add one thing, and that is that unnest unnests all levels of an array:
select * from unnest(ARRAY[ARRAY[1,2,3],Array[5,6,7]]);
unnest
--------
1
2
3
5
6
7
(6 rows)
If you want to loop through one level of a multi-dimensional array, you should loop through the following instead
FOR out_count IN
array_lower(in_transactions, 1) ..
array_upper(in_transactions, 1)
LOOP
-- Fill the bulk payments table
INSERT INTO bulk_payments_in(id, amount)
VALUES (in_transactions[out_count][1],
in_transactions[out_count][2]);
END LOOP;

Related

How can I query a custom datatype object inside an array of said custom datatype in PL/pgSQL?

Suppose I have:
CREATE TYPE compfoo AS (f1 int, f2 text);
And I create a table foo containing two columns: fooid and fooname, corresponding to the fields of compfoo, later I insert some records 1, aa, 2, bb, 3, cc
Then, I define a PL/pgSQL function (more or less as follows:)
create or replace function foo_query()
returns text
language plpgsql
as $$
declare
r compfoo;
arr compfoo [];
footemp compfoo;
result text;
begin
for r in
select * from foo where fooid = 1 OR fooid = 2
loop
arr := array_append(arr, r);
end loop;
foreach footemp in array arr
loop
select footemp.f1 into result where footemp.f1 = 1;
end loop;
return result;
end;
$$
Where I query first foo by column names and save the results into arr, an array of compfoo. Later, I iterate over arr and try to query the elements by their fieldnames as defined in compfoo.
I don't get an error per se in Postgres but the result of my function is null.
What am I doing wrong?
The RAISE NOTICE should be your best friend. You can print the result of some variables at some points of your code. The basic issue are not well initialized values. The arr variable is initialized by NULL value, and any operation over NULL is NULL again.
Another problem is in select footemp.f1 into result where footemp.f1 = 1; statement. SELECT INTO in Postgres overwrite the target variable by NULL value when an result is empty. In second iteration, the result of this query is empty set, and the result variable is set on NULL.
The most big problem of your example is style of programming. You use ISAM style, and your code can be terrible slow.
Don't use array_append in cycle, when you can use array_agg function in query, and you don't need cycle,
Don't use SELECT INTO when you don't read data from tables,
Don't try to repeat Oracle' pattern BULK COLLECT and FOREACH read over collection. PostgreSQL is not Oracle, uses very different architecture, and this pattern doesn't increase performance (like on Oracle), but probably you will lost some performance.
Your fixed code can looks like:
CREATE OR REPLACE FUNCTION public.foo_query()
RETURNS text
LANGUAGE plpgsql
AS $function$
declare
r compfoo;
arr compfoo [] default '{}'; --<<<
footemp compfoo;
result text;
begin
for r in
select * from foo where fooid = 1 or fooid = 2
loop
arr := array_append(arr, r);
end loop;
foreach footemp in array arr
loop
if footemp.f1 = 1 then --<<<
result := footemp.f1;
end if;
end loop;
return result;
end;
$function$
postgres-# ;
It returns expected result. But it is perfect example how don't write stored procedures. Don't try to replace SQL in stored procedures. All code of this procedure can be replaced just by one query. In the end this code can be very slow on bigger data.

postgres plpgsql how to properly convert function to use FORMAT in DECLARE

I am writing a function in POSTGRES v13.3 that when passed an array of column names returns an array of JSONB objects each with the distinct values of one of the columns. I have an existing script that I wish to refactor using FORMAT in the declaration portion of the function.
The existing and working function looks like below. It is passed an array of columns and a dbase name. The a loop presents each column name to an EXECUTE statement that uses JSONB_AGG on the distinct values in the column, creates a JSONB object, and appends that to an array. The array is returned on completion. This is the function:
CREATE OR REPLACE FUNCTION foo1(text[], text)
RETURNS text[] as $$
declare
col text;
interim jsonb;
temp jsonb;
y jsonb[];
begin
foreach col in array $1
loop
execute
'select jsonb_agg(distinct '|| col ||') from ' || $2 into interim;
temp := jsonb_build_object(col, interim);
y := array_append(y,temp);
end loop;
return y;
end;
$$ LANGUAGE plpgsql;
I have refactored the function to the following. The script is now in the DECLARE portion of the function.
CREATE OR REPLACE FUNCTION foo2(_cols text[], _db text)
RETURNS jsonb[]
LANGUAGE plpgsql as
$func$
DECLARE
_script text := format(
'select jsonb_agg( distinct $1) from %1$I', _db
);
col text;
interim jsonb;
temp jsonb;
y jsonb[];
BEGIN
foreach col in array _cols
loop
EXECUTE _script USING col INTO interim;
temp := jsonb_build_object(col, interim);
y := array_append(y,temp);
end loop;
return y;
END
$func$;
Unfortunately the two functions give different results on a toy data set (see bottom):
Original: {"{\"id\": [1, 2, 3]}","{\"val\": [1, 2]}"}
Refactored: {{"id": ["id"]},{"val": ["val"]}}
Here is a db<>fiddle of the preceding.
The challenge is in the EXECUTE. In the first instance the col argument is treated as a column identifier. In the refactored function it seems to be treated as just a text string. I think my approach is consistent with the docs and tutorials (example), and the answer from this forum here and the links included therein. I have tried playing around with combinations of ", ', and || but those were unsuccessful and don't make sense in a format statement.
Where should I be looking for the error in my use of FORMAT?
NOTE 1: From the docs I have so possibly the jsonagg() and distinct are what's preventing the behaviour I want:
Another restriction on parameter symbols is that they only work in SELECT, INSERT, UPDATE, and DELETE commands. In other statement types (generically called utility statements), you must insert values textually even if they are just data values.
TOY DATA SET:
drop table if exists example;
create temporary table example(id int, str text, val integer);
insert into example values
(1, 'a', 1),
(2, 'a', 2),
(3, 'b', 2);
https://www.postgresql.org/docs/14/plpgsql-statements.html#PLPGSQL-STATEMENTS-SQL-ONEROW
The command string can use parameter values, which are referenced in
the command as $1, $2, etc. These symbols refer to values supplied in
the USING clause.
What you want is paramterize sql identifier(column name).
You cannot do that. Access column using variable instead of explicit column name
Which means that select jsonb_agg( distinct $1) from %1$I In here "$1" must be %I type. USING Expression in the manual (EXECUTE command-string [ INTO [STRICT] target ] [ USING expression [, ... ] ];) will pass the literal value to it. But it's valid because select distinct 'hello world' from validtable is valid.
select jsonb_agg( distinct $1) from %1$I In here $1 must be same type as %1$I namely-> sql identifier.
--
Based on the following debug code, then you can solve your problem:
CREATE OR REPLACE FUNCTION foo2(_cols text[], _db text)
RETURNS void
LANGUAGE plpgsql as
$func$
DECLARE
col text;
interim jsonb;temp jsonb; y jsonb[];
BEGIN
foreach col in array _cols
loop
EXECUTE format( 'select jsonb_agg( distinct ( %1I ) ) from %2I', col,_db) INTO interim;
raise info 'interim: %', interim;
temp := jsonb_build_object(col, interim);
raise info 'temp: %', temp;
y := array_append(y,temp);
raise info 'y: %',y;
end loop;
END
$func$;

An error related to arrays in postgresql "ERROR: malformed record literal:"

I am trying to get values from a table into an array and the array definition is different from the table's column definition. I have tried do cast but it is not working. Basically I need as an array (tab_small_str) the values in the table. Can someone please suggest on it:
CREATE TYPE tab_small_str AS (
str CHARACTER VARYING(50)
);
create table test_emp(emp_id integer, ename character varying (10));
insert into test_emp values(1,'a1')
insert into test_emp values(2,'a2')
insert into test_emp values(3,'a3')
CREATE OR REPLACE function test_fn () RETURNS VARCHAR[] as
$$
DECLARE
v_ename tab_small_str[];
i tab_small_str;
BEGIN
SELECT ARRAY(SELECT ename::tab_small_str FROM test_emp) INTO v_ename;
RAISE INFO 'array is: %',v_ename;
RETURN v_ename;
FOREACH i IN ARRAY v_ename
LOOP
RAISE info 'value of ename is%', i;
END LOOP;
END;
$$
language plpgsql;
(function compiles fine).
select test_fn()
--gives below error
ERROR: malformed record literal: "a1"
DETAIL: Missing left parenthesis.
CONTEXT: SQL statement "SELECT ARRAY(SELECT ename::tab_small_str FROM test_emp)"
PL/pgSQL function test_fn() line 7 at SQL statement
********** Error **********
ERROR: malformed record literal: "a1"
SQL state: 22P02
Detail: Missing left parenthesis.
Context: SQL statement "SELECT ARRAY(SELECT ename::tab_small_str FROM test_emp)"
PL/pgSQL function test_fn() line 7 at SQL statement
Hi 404,
i modified as suggested:
CREATE OR REPLACE function test_fn () RETURNS tab_small_str[] as
$$
DECLARE
v_ename tab_small_str[];
i tab_small_str;
BEGIN
SELECT ARRAY(SELECT ROW(ename)::tab_small_str FROM test_emp) INTO v_ename;
RAISE INFO '%',v_ename;
FOREACH i IN ARRAY v_ename
LOOP
RAISE NOTICE '%', i;
END LOOP;
RETURN v_ename;
END;
$$
language plpgsql;
it returns output as:
INFO: {(a1),(a2),(a3)}
CONTEXT: PL/pgSQL function test_fn() line 9 at RAISE
NOTICE: (a1)
CONTEXT: PL/pgSQL function test_fn() line 13 at RAISE
NOTICE: (a2)
CONTEXT: PL/pgSQL function test_fn() line 13 at RAISE
NOTICE: (a3)
CONTEXT: PL/pgSQL function test_fn() line 13 at RAISE
My question is why the output is surrounded by bracket - why not just a1 but (a1). Can you please suggest on it?
Your new type is not a "single field data type", for want of a better description, where you can cast something like a VARCHAR(10) directly to it; it's a ROW containing a single field. So something like 'blah'::tab_small_str fails because it's trying to cast that text to the type which contains a field, rather than the field itself.
To resolve, using your existing query:
SELECT ename::tab_small_str FROM test_emp
Change to:
SELECT ROW(ename)::tab_small_str FROM test_emp
As to why your results are surrounded by brackets: that is how a ROW or composite type is displayed when shown as a single field (or, non-expanded): for example, if you do SELECT * FROM test_emp, the * returns all fields individually as separate columns; however if you do SELECT test_emp FROM test_emp, that will return the table row unexpanded, so it will look like so:
(1,a1)
(2,a2)
(3,a3)
And composite types are exactly the same. i tab_small_str; - think of i as test_emp, which contains fields which can be expanded. In your code you are printing the object i, rather than i.* or i.str. So change your code to:
FOREACH i IN ARRAY v_ename
LOOP
RAISE NOTICE '%', i.str;
END LOOP;

How to make a loop structure out of an array in plpgsql? [duplicate]

In plpgsql, I want to get the array contents one by one from a two dimension array.
DECLARE
m varchar[];
arr varchar[][] := array[['key1','val1'],['key2','val2']];
BEGIN
for m in select arr
LOOP
raise NOTICE '%',m;
END LOOP;
END;
But the above code returns:
{{key1,val1},{key2,val2}}
in one line. I want to be able to loop over and call another function which takes parameters like:
another_func(key1,val1)
Since PostgreSQL 9.1
There is the convenient FOREACH which can loop over slices of arrays. The manual:
The target variable must be an array, and it receives successive
slices of the array value, where each slice is of the number of
dimensions specified by SLICE.
DO
$do$
DECLARE
m text[];
arr text[] := '{{key1,val1},{key2,val2}}'; -- array literal
BEGIN
FOREACH m SLICE 1 IN ARRAY arr
LOOP
RAISE NOTICE 'another_func(%,%)', m[1], m[2];
END LOOP;
END
$do$;
db<>fiddle here - with a function printing results, instead of DO
LANGUAGE plpgsql is the default for a DO statement so we can omit the declaration.
There is no difference between text[] and text[][] for the Postgres type system. See:
Initial array in function to aggregate multi-dimensional array
Postgres 9.0 or older
DO
$do$
DECLARE
arr text[] := array[['key1','val1'],['key2','val2']]; -- array constructor
BEGIN
FOR i IN array_lower(arr, 1) .. array_upper(arr, 1)
LOOP
RAISE NOTICE 'another_func(%,%)', arr[i][1], arr[i][2];
END LOOP;
END
$do$;

Postgresql transform multidimensional array to relational model

I have seeing this example multiple times but only on a single column
INSERT INTO user_subservices(user_id, subservice_id)
SELECT 1 id, x
FROM unnest(ARRAY[1,2,3,4,5,6,7,8,22,33]) x
I need to insert multiple columns with multidimensional array like:
INSERT INTO items(order_id,lot_serial,pallets,oum) VALUES from Array
Array example:
[[3,'Fff767',89,'Boxes'],[3,'FDH6784',45,'Boxes'],[3,'FDH6788',97,'Boxes']...]
What would be the approach to do this with Postgresql 9.4.5
You can create reduce_dim function to unnest ultidimensional array.
See https://wiki.postgresql.org/wiki/Unnest_multidimensional_array
CREATE OR REPLACE FUNCTION public.reduce_dim(anyarray)
RETURNS SETOF anyarray AS
$function$
DECLARE
s $1%TYPE;
BEGIN
FOREACH s SLICE 1 IN ARRAY $1 LOOP
RETURN NEXT s;
END LOOP;
RETURN;
END;
$function$
LANGUAGE plpgsql IMMUTABLE;
insert into items select t[1],t[2],t[3],t[4] from (select reduce_dim(reduce_dim(ARRAY[['3','Fff767','89','Boxes'],['3','FDH6784','45','Boxes'],['3','FDH6788','97','Boxes']])) t) as y;