Selecting from multidimensional array parameter in Postgres - postgresql

I have a function that does an insert in a parent table then inserts into a child table. The last parameter is an array of variable length. I use this to do the insert into the child table by sending an array of comma delimited pairs and parsing these out.
create or replace function neil_test(time_stamp timestamp with time zone, type_id text, raw_message text,field_values text[])
returns void
AS $$
DECLARE
last_message_id bigint;
x text;
BEGIN
INSERT INTO message(time_stamp,type_id,raw_message) values(time_stamp,type_id,raw_message);
select into last_message_id currval(pg_get_serial_sequence('message', 'id'));
foreach x in ARRAY field_values
LOOP
insert into message_field_value(last_message_id,field_id,fieldValue) select left(x,strpos(x,',')-1), right(x,length(x)-strpos(x,','));
END LOOP;
END
$$LANGUAGE plpgsql
It's called like this:
select neil_test('2001-01-01 08:00:00.1234','F','RAW',ARRAY['One,value1','two,value2','three,value3'])
This works OK but what I'd really like to do is use the array directly. Something like :
select neil_test('2001-01-01 08:00:00.1234','F',
'RAW',ARRAY[['One', 'value1'],['two','value2'],['three','value3']])
...
insert into message_field_value(last_message_id,fieldid, field value)
select field_values[1], field_values[2]
I've tried the unnest function but this doesn't work as it seems to flatten out the whole array and I lose the pairs. Is something like this even possible with Postgres arrays?

Use SLICE in the FOREACH statement to iterate over elements of multidimencional array. Without SLICE it iterates over individual elements of the array. This works for me:
CREATE OR REPLACE FUNCTION neil_test2(time_stamp timestamp with time zone, type_id text, raw_message text,field_values text[][2])
RETURNS void
AS $$
DECLARE
last_message_id bigint;
x TEXT[];
BEGIN
INSERT INTO message(time_stamp,type_id,raw_message) VALUES (time_stamp,type_id,raw_message);
SELECT INTO last_message_id currval(pg_get_serial_sequence('message', 'id'));
FOREACH x SLICE 1 in ARRAY field_values LOOP
INSERT INTO message_field_value(last_message_id, field_id, field_value) SELECT last_message_id, x[1], x[2];
END LOOP;
END
$$LANGUAGE plpgsql;

Related

"cannot assign non-composite value to a record variable " while storing result in array

on Postgres 13 I have a trigger execute ON UPDATE of a table. In this trigger I want to store in an array a result from a query, because I will need to use it inside an inner iterator, multiple times. In this way I avoid performing the same query at every iteration and I can reuse the array.
At first I tried like this, I typed the variable as array record:
create or REPLACE FUNCTION check_companyprofile_between_records_constraints_trigger_hook()
RETURNS trigger as
$body$
declare
l_table1_results record[];
l_table1_result record;
l_record record;
begin
l_table1_results := ARRAY(
select
date_start,
coalesce(date_end, 'infinity'::date) as date_end
FROM application.section_profile_company_metadata
WHERE persona_id = NEW.persona_id and id != new.id
);
for l_record in (
select foo,bar,baz
from table2
)
loop
foreach l_table1_result in array l_table1_results
loop
-- Check some stuffs
end loop;
end loop;
RETURN NEW;
end
$body$
LANGUAGE plpgsql;
But I'm getting a
Errore SQL [0A000]: ERROR: variable "l_table1_results" has pseudo-type record[]
Doing some research I discovered it is not possible to assign an in-memory variable with the result of a query, but instead I need to use a custom type. Ok, so I tried also this
create type apc_dates_pair as (
date_start date,
date_end date
);
create or REPLACE FUNCTION check_companyprofile_between_records_constraints_trigger_hook()
RETURNS trigger as
$body$
declare
l_table1_results apc_dates_pair[];
l_table1_result record;
l_record record;
begin
l_table1_results := ARRAY(
select
date_start,
coalesce(date_end, 'infinity'::date) as date_end
FROM application.section_profile_company_metadata
WHERE persona_id = NEW.persona_id and id != new.id
);
-- TODO do some stuffs with array
for l_record in (
select foo,bar,baz
from table2
)
loop
foreach l_table1_result in l_table1_results
loop
-- Check some stuffs
end loop;
end loop;
RETURN NEW;
end
$body$
LANGUAGE plpgsql;
create or REPLACE FUNCTION check_companyprofile_between_records_constraints_trigger_hook()
RETURNS trigger as
$body$
declare
l_table1_results apc_dates_pair[];
l_table1_result record;
l_record record;
begin
l_table1_results := ARRAY(
select row(
date_start,
coalesce(date_end, 'infinity'::date)
)
FROM application.section_profile_company_metadata
WHERE persona_id = NEW.persona_id and id != new.id
);
for l_record in (
select foo,bar,baz
from table2
)
loop
foreach l_table1_result in array l_table1_results
loop
-- Check some stuffs
end loop;
end loop;
RETURN NEW;
end
$body$
LANGUAGE plpgsql;
Different error, but still an error:
cannot assign non-composite value to a record variable
And from this I did not find much.
Is it possible at all to store a temp query inside and array of records and then iterate them?
An explanation more then an answer:
You seem to be confusing composite types with arrays, they are different things in Postgres. Most notably a Postgres Array contains values of a single type. You can have int[] all integers, varchar[] all characters, and so on, but you cannot mix types in a single array. A Composite Type on the other hand can do just that, have included values that are of different types. In plpgsql declaring a record is to basically declare an anonymous composite type that acquires the properties of whatever actual composite type is passed to it. This is why you get your second error; "cannot assign non-composite value to a record variable". You are trying to assign an array to a composite type, in this case a record. Given that the values you want to store are both dates you could dispense with create type apc_dates_pair ... and just do l_table1_results date[]. Then iterate over the array using the form shown at Array loop.

postgres plpgsql how to properly convert function to use FORMAT in DECLARE

I am writing a function in POSTGRES v13.3 that when passed an array of column names returns an array of JSONB objects each with the distinct values of one of the columns. I have an existing script that I wish to refactor using FORMAT in the declaration portion of the function.
The existing and working function looks like below. It is passed an array of columns and a dbase name. The a loop presents each column name to an EXECUTE statement that uses JSONB_AGG on the distinct values in the column, creates a JSONB object, and appends that to an array. The array is returned on completion. This is the function:
CREATE OR REPLACE FUNCTION foo1(text[], text)
RETURNS text[] as $$
declare
col text;
interim jsonb;
temp jsonb;
y jsonb[];
begin
foreach col in array $1
loop
execute
'select jsonb_agg(distinct '|| col ||') from ' || $2 into interim;
temp := jsonb_build_object(col, interim);
y := array_append(y,temp);
end loop;
return y;
end;
$$ LANGUAGE plpgsql;
I have refactored the function to the following. The script is now in the DECLARE portion of the function.
CREATE OR REPLACE FUNCTION foo2(_cols text[], _db text)
RETURNS jsonb[]
LANGUAGE plpgsql as
$func$
DECLARE
_script text := format(
'select jsonb_agg( distinct $1) from %1$I', _db
);
col text;
interim jsonb;
temp jsonb;
y jsonb[];
BEGIN
foreach col in array _cols
loop
EXECUTE _script USING col INTO interim;
temp := jsonb_build_object(col, interim);
y := array_append(y,temp);
end loop;
return y;
END
$func$;
Unfortunately the two functions give different results on a toy data set (see bottom):
Original: {"{\"id\": [1, 2, 3]}","{\"val\": [1, 2]}"}
Refactored: {{"id": ["id"]},{"val": ["val"]}}
Here is a db<>fiddle of the preceding.
The challenge is in the EXECUTE. In the first instance the col argument is treated as a column identifier. In the refactored function it seems to be treated as just a text string. I think my approach is consistent with the docs and tutorials (example), and the answer from this forum here and the links included therein. I have tried playing around with combinations of ", ', and || but those were unsuccessful and don't make sense in a format statement.
Where should I be looking for the error in my use of FORMAT?
NOTE 1: From the docs I have so possibly the jsonagg() and distinct are what's preventing the behaviour I want:
Another restriction on parameter symbols is that they only work in SELECT, INSERT, UPDATE, and DELETE commands. In other statement types (generically called utility statements), you must insert values textually even if they are just data values.
TOY DATA SET:
drop table if exists example;
create temporary table example(id int, str text, val integer);
insert into example values
(1, 'a', 1),
(2, 'a', 2),
(3, 'b', 2);
https://www.postgresql.org/docs/14/plpgsql-statements.html#PLPGSQL-STATEMENTS-SQL-ONEROW
The command string can use parameter values, which are referenced in
the command as $1, $2, etc. These symbols refer to values supplied in
the USING clause.
What you want is paramterize sql identifier(column name).
You cannot do that. Access column using variable instead of explicit column name
Which means that select jsonb_agg( distinct $1) from %1$I In here "$1" must be %I type. USING Expression in the manual (EXECUTE command-string [ INTO [STRICT] target ] [ USING expression [, ... ] ];) will pass the literal value to it. But it's valid because select distinct 'hello world' from validtable is valid.
select jsonb_agg( distinct $1) from %1$I In here $1 must be same type as %1$I namely-> sql identifier.
--
Based on the following debug code, then you can solve your problem:
CREATE OR REPLACE FUNCTION foo2(_cols text[], _db text)
RETURNS void
LANGUAGE plpgsql as
$func$
DECLARE
col text;
interim jsonb;temp jsonb; y jsonb[];
BEGIN
foreach col in array _cols
loop
EXECUTE format( 'select jsonb_agg( distinct ( %1I ) ) from %2I', col,_db) INTO interim;
raise info 'interim: %', interim;
temp := jsonb_build_object(col, interim);
raise info 'temp: %', temp;
y := array_append(y,temp);
raise info 'y: %',y;
end loop;
END
$func$;

In clause in postgres

Need Output from table with in clause in PostgreSQL
I tried to make loop or ids passed from my code. I did same to update the rows dynamically, but for select I m not getting values from DB
CREATE OR REPLACE FUNCTION dashboard.rspgetpendingdispatchbyaccountgroupidandbranchid(
IN accountgroupIdCol numeric(8,0),
IN branchidcol character varying
)
RETURNS void
AS
$$
DECLARE
ArrayText text[];
i int;
BEGIN
select string_to_array(branchidcol, ',') into ArrayText;
i := 1;
loop
if i > array_upper(ArrayText, 1) then
exit;
else
SELECT
pd.branchid,pd.totallr,pd.totalarticle,pd.totalweight,
pd.totalamount
FROM dashboard.pendingdispatch AS pd
WHERE
pd.accountgroupid = accountgroupIdCol AND pd.branchid IN(ArrayText[i]::numeric);
i := i + 1;
end if;
END LOOP;
END;
$$ LANGUAGE 'plpgsql' VOLATILE;
There is no need for a loop (or PL/pgSQL actually)
You can use the array directly in the query, e.g.:
where pd.branchid = any (string_to_array(branchidcol, ','));
But your function does not return anything, so obviously you won't get a result.
If you want to return the result of that SELECT query, you need to define the function as returns table (...) and then use return query - or even better make it a SQL function:
CREATE OR REPLACE FUNCTION dashboard.rspgetpendingdispatchbyaccountgroupidandbranchid(
IN accountgroupIdCol numeric(8,0),
IN branchidcol character varying )
RETURNS table(branchid integer, totallr integer, totalarticle integer, totalweight numeric, totalamount integer)
AS
$$
SELECT pd.branchid,pd.totallr,pd.totalarticle,pd.totalweight, pd.totalamount
FROM dashboard.pendingdispatch AS pd
WHERE pd.accountgroupid = accountgroupIdCol
AND pd.branchid = any (string_to_array(branchidcol, ',')::numeric[]);
$$
LANGUAGE sql
VOLATILE;
Note that I guessed the data types for the columns of the query based on their names. You have to adjust the line with returns table (...) to match the data types of the select columns.

Postgresql transform multidimensional array to relational model

I have seeing this example multiple times but only on a single column
INSERT INTO user_subservices(user_id, subservice_id)
SELECT 1 id, x
FROM unnest(ARRAY[1,2,3,4,5,6,7,8,22,33]) x
I need to insert multiple columns with multidimensional array like:
INSERT INTO items(order_id,lot_serial,pallets,oum) VALUES from Array
Array example:
[[3,'Fff767',89,'Boxes'],[3,'FDH6784',45,'Boxes'],[3,'FDH6788',97,'Boxes']...]
What would be the approach to do this with Postgresql 9.4.5
You can create reduce_dim function to unnest ultidimensional array.
See https://wiki.postgresql.org/wiki/Unnest_multidimensional_array
CREATE OR REPLACE FUNCTION public.reduce_dim(anyarray)
RETURNS SETOF anyarray AS
$function$
DECLARE
s $1%TYPE;
BEGIN
FOREACH s SLICE 1 IN ARRAY $1 LOOP
RETURN NEXT s;
END LOOP;
RETURN;
END;
$function$
LANGUAGE plpgsql IMMUTABLE;
insert into items select t[1],t[2],t[3],t[4] from (select reduce_dim(reduce_dim(ARRAY[['3','Fff767','89','Boxes'],['3','FDH6784','45','Boxes'],['3','FDH6788','97','Boxes']])) t) as y;

Create a function to get column from multiple tables in PostgreSQL

I'm trying to create a function to get a field value from multiple tables in my database. I made script like this:
CREATE OR REPLACE FUNCTION get_all_changes() RETURNS SETOF RECORD AS
$$
DECLARE
tblname VARCHAR;
tblrow RECORD;
row RECORD;
BEGIN
FOR tblrow IN SELECT tablename FROM pg_catalog.pg_tables WHERE schemaname='public' LOOP /*FOREACH tblname IN ARRAY $1 LOOP*/
RAISE NOTICE 'r: %', tblrow.tablename;
FOR row IN SELECT MAX("lastUpdate") FROM tblrow.tablename LOOP
RETURN NEXT row;
END LOOP;
END LOOP;
END
$$
LANGUAGE 'plpgsql' ;
SELECT get_all_changes();
But it is not working, everytime it shows this error
tblrow.tablename" not defined in line "FOR row IN SELECT MAX("lastUpdate") FROM tblrow.tablename LOOP"
Your inner FOR loop must use the FOR...EXECUTE syntax as shown in the manual:
FOR target IN EXECUTE text_expression [ USING expression [, ... ] ] LOOP
statements
END LOOP [ label ];
In your case something along this line:
FOR row IN EXECUTE 'SELECT MAX("lastUpdate") FROM ' || quote_ident(tblrow.tablename) LOOP
RETURN NEXT row;
END LOOP
The reason for this is explained in the manual somewhere else:
Oftentimes you will want to generate dynamic commands inside your PL/pgSQL functions, that is, commands that will involve different tables or different data types each time they are executed. PL/pgSQL's normal attempts to cache plans for commands (as discussed in Section 39.10.2) will not work in such scenarios. To handle this sort of problem, the EXECUTE statement is provided[...]
Answer to your new question (mislabeled as answer):
This can be much simpler. You do not need to create a table just do define a record type.
If at all, you would better create a type with CREATE TYPE, but that's only efficient if you need the type in multiple places. For just a single function, you can use RETURNS TABLE instead :
CREATE OR REPLACE FUNCTION get_all_changes(text[])
RETURNS TABLE (tablename text
,"lastUpdate" timestamp with time zone
,nums integer) AS
$func$
DECLARE
tblname text;
BEGIN
FOREACH tblname IN ARRAY $1 LOOP
RETURN QUERY EXECUTE format(
$f$SELECT '%I', MAX("lastUpdate"), COUNT(*)::int FROM %1$I
$f$, tblname)
END LOOP;
END
$func$ LANGUAGE plpgsql;
A couple more points:
Use RETURN QUERY EXECUTE instead of the nested loop. Much simpler and faster.
Column aliases would only serve as documentation, those names are discarded in favor of the names declared in the RETURNS clause (directly or indirectly).
Use format() with %I to replace the concatenation with quote_ident() and %1$I to refer to the same parameter another time.
count() usually returns type bigint. Cast the integer, since you defined the column in the return type as such: count(*)::int.
Thanks,
I finally made my script like:
CREATE TABLE IF NOT EXISTS __rsdb_changes (tablename text,"lastUpdate" timestamp with time zone, nums bigint);
CREATE OR REPLACE FUNCTION get_all_changes(varchar[]) RETURNS SETOF __rsdb_changes AS /*TABLE (tablename varchar(40),"lastUpdate" timestamp with time zone, nums integer)*/
$$
DECLARE
tblname VARCHAR;
tblrow RECORD;
row RECORD;
BEGIN
FOREACH tblname IN ARRAY $1 LOOP
/*RAISE NOTICE 'r: %', tblrow.tablename;*/
FOR row IN EXECUTE 'SELECT CONCAT('''|| quote_ident(tblname) ||''') AS tablename, MAX("lastUpdate") AS "lastUpdate",COUNT(*) AS nums FROM ' || quote_ident(tblname) LOOP
/*RAISE NOTICE 'row.tablename: %',row.tablename;*/
/*RAISE NOTICE 'row.lastUpdate: %',row."lastUpdate";*/
/*RAISE NOTICE 'row.nums: %',row.nums;*/
RETURN NEXT row;
END LOOP;
END LOOP;
RETURN;
END
$$
LANGUAGE 'plpgsql' ;
Well, it works. But it seems I can only create a table to define the return structure instead of just RETURNS SETOF RECORD. Am I right?
Thanks again.