Data Type for select in plpgsql function and access its fields - postgresql

I have the following tables in a Postgres 9.5 database:
product
Column | Type | Modifiers
----------------+-----------------------------+-----------------------------------------------------
id | integer | not null default nextval('product_id_seq'::regclass)
name | character varying(100) |
number_of_items | integer |
created_at | timestamp without time zone | default now()
updated_at | timestamp without time zone | default now()
total_number | integer |
provider_id | integer |
Indexes:
"pk_product" PRIMARY KEY, btree (id)
Foreign-key constraints:
"fk_product_provider" FOREIGN KEY (provider_id) REFERENCES provider(id)
And we also have
provider
Column | Typ | Modifiers
-------------+------------------------+------------------------------
id | integer | not null default nextval('property_id_seq'::regclass)
name | text |
description | text |
created_at | timestamp without time zone | default now()
updated_at | timestamp without time zone | default now()
Indexes:
"pk_provider" PRIMARY KEY, btree (id)
I am implelemtnig a plpgsql function which is supposed to find some specific products of a provider and loop through them
products = select u_id, number_of_items from product
where provider_id = p_id and total_number > limit;
loop
//here I need to loop through the products
end loop;
Question
what kind of data type should I declare for the products variables in order to store queried products into it? and also how should I have later on access to its columns like id or number_of_items?

In PostgreSQL, creating table also defines a composite data type with the same name as the table.
You can either use a variable of that type:
DECLARE
p product;
BEGIN
FOR p IN SELECT product FROM product WHERE ...
LOOP
[do something with "p.id" and "p.val"]
END LOOP;
END;
Or you can use several variables for the individual fields you need (probably better):
DECLARE
v_id integer;
v_val text;
BEGIN
FOR v_id, v_val IN SELECT id, val FROM product WHERE ...
LOOP
[do something with "v_id" and "v_val"]
END LOOP;
END;

Related

Duplicate key found, but nothing matches in the table

I have a function which receives some parameters, does a SELECT to see if a table row exists and returns FALSE if so. If not, it does an INSERT but currently always fails with a 'duplicate key'. Here's a pseudo-code version...
CREATE OR REPLACE FUNCTION bob_function (
p_work_id INTEGER,
p_location_id VARCHAR,
p_area_id VARCHAR,
p_scheduled_work_no INTEGER,
p_start_date_time TIMESTAMPTZ,
p_work_date_time TIMESTAMPTZ,
p_user_id INTEGER,
p_comments TEXT,
p_work_stat_code CHAR(1)
)
RETURNS BOOLEAN AS $$
BEGIN
IF EXISTS (
SELECT 1
FROM work_table
WHERE location_id = p_location_id
AND area_id = p_area_id
AND work_id = p_work_id
AND scheduled_work_no = p_scheduled_work_no
AND start_date_time = p_start_date_time
AND user_work_id = p_user_id
AND work_date_time = p_work_date_time
)
THEN
RAISE NOTICE 'Work already exists - SKIPPING';
RETURN FALSE;
END IF;
INSERT INTO work_table (
location_id,
area_id,
work_id,
scheduled_work_no,
start_date_time,
user_work_id,
work_date_time,
stat_code,
comment
)
VALUES (
p_location_id,
p_area_id,
p_work_id,
p_scheduled_work_no,
p_start_date_time,
p_user_id,
p_work_date_time,
v_work_stat_code,
p_comments
);
RETURN TRUE;
END;
$$ LANGUAGE plpgsql SECURITY DEFINER;
The primary key is defined thus...
myDb=# \d task_work_pk
Index "schema1.task_work_pk"
Column | Type | Key? | Definition
-------------------+-----------------------------+------+-------------------
location_id | character varying(8) | yes | location_id
area_id | character varying(3) | yes | area_id
work_id | integer | yes | work_id
scheduled_work_no | integer | yes | scheduled_work_no
start_date_time | timestamp(0) with time zone | yes | start_date_time
user_work_id | integer | yes | user_work_id
work_date_time | timestamp(0) with time zone | yes | work_date_time
primary key, btree, for table "schema1.work_table"
Currently I get the following error every time I run this function...
ERROR: 23505: duplicate key value violates unique constraint "task_work_pk"
DETAIL: Key (location_id, area_id, work_id, scheduled_work_no, start_date_time, user_work_id, work_date_time)=(SITE_1, BOB, 218, 5, 2021-07-09 00:28:00+10, 1, 2021-07-09 21:00:15+10) already exists.
There are no rows whatsoever with work_id = 218 and this is the only place in the entire database where this table is written to. The function is only called no more frequently than once a minute and I'm 99% sure I've not got any race condition.
EDIT: updated to remove errors
I'm ignoring your PLPGSQL code because it is not real code and has obvious flaws.
Given that 218 doesn't exist the only way to cause that error without 218 pre-existing is to insert the same record twice in a single transaction.

postgres default values are applying on update, not just at create

In the process of writing this question, I found the answer and will post it below. If there are already duplicates of this question, please notify me and I will remove it, I was unable to find any.
I've got two columns tracking changes made to a table in postgres:
created_at timestamp default now()
updated_at timestamp
the updated_at column is being updated by a trigger:
united_states_congress=> \d congressional_bill_summaries;
Table "public.congressional_bill_summaries"
Column | Type | Modifiers
-------------+-----------------------------+---------------------------------------------------------------------------
id | bigint | not null default nextval('congressional_bill_summaries_id_seq'::regclass)
text | text |
created_at | timestamp without time zone | default now()
updated_at | timestamp without time zone |
bill_kid | integer | not null
date | date | not null
description | character varying(255) | not null
text_hash | uuid |
Indexes:
"congressional_bill_summaries_pkey" PRIMARY KEY, btree (id)
"congressional_bill_summaries_bill_kid_date_description_key" UNIQUE CONSTRAINT, btree (bill_kid, date, description)
Triggers:
hash_all_the_things BEFORE INSERT ON congressional_bill_summaries FOR EACH ROW EXECUTE PROCEDURE hash_this_foo()
update_iz_yoo BEFORE UPDATE ON congressional_bill_summaries FOR EACH ROW EXECUTE PROCEDURE update_iz_now()
as is one other column of the table, text_hash
My expected behavior is that when a line is first inserted, the created_at column will update to default value (which I understand to be the time at which the current transaction began, not the time of the specific query).
My expected behavior is that when a line is updated, the updated_at line will be updated by this function:
CREATE OR REPLACE FUNCTION public.update_iz_now()
RETURNS trigger
LANGUAGE plpgsql
AS $function$
BEGIN
NEW.updated_at = now();
RETURN NEW;
END;
$function$
but that created_at will remain unchanged, because a value is already in the column, so it should not override with the default.
created_at is initially functioning correctly:
united_states_congress=> select created_at, updated_at from congressional_bill_actions limit 5;
created_at | updated_at
----------------------------+------------
2017-01-28 00:08:11.238773 |
2017-01-28 00:08:11.255533 |
2017-01-28 00:08:15.036168 |
2017-01-28 00:08:15.047991 |
2017-01-28 00:08:15.071715 |
(5 rows)
But then when a line is updated, created_at is being changed to match the insert value of updated_at, leaving me with:
united_states_congress=> select created_at, updated_at from congressional_bill_actions where updated_at is not null limit 5;
created_at | updated_at
----------------------------+----------------------------
2017-01-28 07:55:34.078783 | 2017-01-28 07:55:34.078783
2017-02-01 18:47:50.673996 | 2017-02-01 18:47:50.673996
2017-02-02 14:50:33.066341 | 2017-02-02 14:50:33.066341
2017-02-02 14:50:33.083343 | 2017-02-02 14:50:33.083343
2017-02-03 13:58:34.950716 | 2017-02-03 13:58:34.950716
(5 rows)
I have been all over the internet trying to figure this one out, but the internet keeps helpfully routing me to questions about "how to create default values" and "how to make triggers."
This obviously must be a usage problem somewhere on my end, but I'm having trouble identifying it. Just in case, here is the other trigger being run on the table (on insert):
CREATE OR REPLACE FUNCTION public.hash_this_foo()
RETURNS trigger
LANGUAGE plpgsql
AS $function$
BEGIN
NEW.text_hash = md5(NEW.text)::uuid;
RETURN NEW;
END;
$function$
In the process of writing this question, I found the answer and will post it below. If there are already duplicates of this question, please notify me and I will remove it, I was unable to find any.
The problem here was my UPSERT handling, during which the schema of the table was being pulled in, resulting in the dynamic creation of queries that included lines like this:
ON CONFLICT ON CONSTRAINT congressional_bill_actions_bill_kid_date_action_actor_key DO UPDATE SET created_at = EXCLUDED.created_at,
because created_at was being set automatically to the EXCLUDED.created_at, this was causing the default value to overwrite the existing one precisely because I was instructing it to do so.
So when writing UPSERT handlers, this is something to be aware of, it would seem.
(Note: the way to avoid this is simply not to pull in any columns where column_default is not null.)

How to use subquery In Sphinx's multi-valued attributes (MVA) for query type

My database is PostgreSQL, and I want to use Sphinx Search Engines to index my data;
How can i use sql_attr_multi to fetch the relationship data?
The tables in postgresql, schemas is:
crm=# \d orders
Table "orders"
Column | Type | Modifiers
-----------------+-----------------------------+----------------------------------------
id | bigint | not null
trade_id | bigint | not null
item_id | bigint | not null
price | numeric(10,2) | not null
total_amount | numeric(10,2) | not null
subject | character varying(255) | not null default ''::character varying
status | smallint | not null default 0
created_at | timestamp without time zone | not null default now()
updated_at | timestamp without time zone | not null default now()
Indexes:
"orders_pkey" PRIMARY KEY, btree (id)
"orders_trade_id_idx" btree (trade_id)
crm=# \d trades
Table "trades"
Column | Type | Modifiers
-----------------------+-----------------------------+---------------------------------------
id | bigint | not null
operator_id | bigint | not null
customer_id | bigint | not null
category_ids | bigint[] | not null
total_amount | numeric(10,2) | not null
discount_amount | numeric(10,2) | not null
created_at | timestamp without time zone | not null default now()
updated_at | timestamp without time zone | not null default now()
Indexes:
"trades_pkey" PRIMARY KEY, btree (id)
The Sphinx's config is:
source trades_src
{
type = pgsql
sql_host = 10.10.10.10
sql_user = ******
sql_pass = ******
sql_db = crm
sql_port = 5432
sql_query = \
SELECT id, operator_id, customer_id, category_ids, total_amount, discount_amount, \
date_part('epoch',created_at) AS created_at, \
date_part('epoch',updated_at) AS updated_at \
FROM public.trades;
#attributes
sql_attr_bigint = operator_id
sql_attr_bigint = customer_id
sql_attr_float = total_amount
sql_attr_float = discount_amount
sql_attr_multi = bigint category_ids from field category_ids
#sql_attr_multi = bigint order_ids from query; SELECT id FROM orders
#how can i add where condition is the query for orders? eg. WHERE trade_id = ?
sql_attr_timestamp = created_at
sql_attr_timestamp = updated_at
}
I used MVA (multi-valued attributes) on category_ids field, and it is the ARRAY type in Postgresql.
But I donot know How to define MVA on order_ids. It will be through subquery?
Copied from sphinx forum....
sql_attr_multi = bigint order_ids from query; SELECT trade_id,id FROM orders ORDER BY trade_id
The first column of the query is the sphinx 'document_id' (ie the id in main sql_query)
The second column is the value to insert into the MVA array for that document.
(The ORDER BY might not strictly be needed, but sphinx is much quicker at processing the data if ordered by document_id IIRC)

postgresql trigger for filling new column on insert

I have a table Table_A:
\d "Table_A";
Table "public.Table_A"
Column | Type | Modifiers
----------+---------+-------------------------------------------------------------
id | integer | not null default nextval('"Table_A_id_seq"'::regclass)
field1 | bigint |
field2 | bigint |
and now I want to add a new column. So I run:
ALTER TABLE "Table_A" ADD COLUMN "newId" BIGINT DEFAULT NULL;
now I have:
\d "Table_A";
Table "public.Table_A"
Column | Type | Modifiers
----------+---------+-------------------------------------------------------------
id | integer | not null default nextval('"Table_A_id_seq"'::regclass)
field1 | bigint |
field2 | bigint |
newId | bigint |
And I want newId to be filled with the same value as id for new/updated rows.
I created the following function and trigger:
CREATE OR REPLACE FUNCTION autoFillNewId() RETURNS TRIGGER AS $$
BEGIN
NEW."newId" := NEW."id";
RETURN NEW;
END $$ LANGUAGE plpgsql;
CREATE TRIGGER "newIdAutoFill" AFTER INSERT OR UPDATE ON "Table_A" EXECUTE PROCEDURE autoFillNewId();
Now if I insert something with:
INSERT INTO "Table_A" values (97, 1, 97);
newId is not filled:
select * from "Table_A" where id = 97;
id | field1 | field2 | newId
----+----------+----------+-------
97 | 1 | 97 |
Note: I also tried with FOR EACH ROW from some answer here in SO
What's missing me?
You need a BEFORE INSERT OR UPDATE ... FOR EACH ROW trigger to make this work:
CREATE TRIGGER "newIdAutoFill"
BEFORE INSERT OR UPDATE ON "Table_A"
FOR EACH ROW EXECUTE PROCEDURE autoFillNewId();
A BEFORE trigger takes place before the new row is inserted or updated, so you can still makes changes to the field values. An AFTER trigger is useful to implement some side effect, like auditing of changes or cascading changes to other tables.
By default, triggers are FOR EACH STATEMENT and then the NEW parameter is not defined (because the trigger does not operate on a row). So you have to specify FOR EACH ROW.

postgresql 8.4 trigger/storedproc error : how to fix it?

An INSERT on a table triggers a stored proc where the following error occurs.
ERROR: column "targetedfamily" is of type boolean but expression is of type character varying
Hint: You will need to rewrite or cast the expression.
Where: PL/pgSQL function "fn_family_audit" line 19 at SQL statement
And here's the ERRING stored proc (notice that my attempt to fix the problem by doing CAST(NEW.targetedfamily AS BOOLEAN) does NOT seem to work)
CREATE OR REPLACE FUNCTION fn_family_audit() RETURNS TRIGGER AS $tr_family_audit$
BEGIN
--
-- Create a row in family_audit to reflect the operation performed on family,
-- make use of the special variable TG_OP to work out the operation.
--
IF (TG_OP = 'DELETE') THEN
INSERT INTO public.family_audit values (
DEFAULT, 'D', OLD.family_id, OLD.familyserialno, OLD.node_id, OLD.sourcetype, OLD.familyname,
OLD.familynamelocallang, OLD.hofname, OLD.hofnamelocallang, OLD.targetedfamily, OLD.homeless,
OLD.landless, OLD.dependentonlabour, OLD.womenprimaryearner, OLD.landlinenumber, OLD.username , now());
RETURN OLD;
ELSIF (TG_OP = 'UPDATE') THEN
INSERT INTO public.family_audit values(
DEFAULT, 'U',NEW.family_id, NEW.familyserialno, NEW.node_id, NEW.sourcetype, NEW.familyname,
NEW.familynamelocallang, NEW.hofname, NEW.hofnamelocallang, NEW.targetedfamily, NEW.homeless,
NEW.landless, NEW.dependentonlabour, NEW.womenprimaryearner, NEW.landlinenumber, NEW.username , now());
RETURN NEW;
ELSIF (TG_OP = 'INSERT') THEN
INSERT INTO public.family_audit values(
DEFAULT, 'I',NEW.family_id, NEW.familyserialno, NEW.node_id, NEW.sourcetype, NEW.familyname,
NEW.familynamelocallang, NEW.hofname, NEW.hofnamelocallang, CAST(NEW.targetedfamily AS BOOLEAN), NEW.homeless,
NEW.landless, NEW.dependentonlabour, NEW.womenprimaryearner, NEW.landlinenumber, NEW.username , now());
RETURN NEW;
END IF;
RETURN NULL; -- result is ignored since this is an AFTER trigger
END;
$tr_family_audit$ LANGUAGE plpgsql;
Here's the table definition
nucleus4=# \d family;
Table "public.family"
Column | Type | Modifiers
---------------------+-----------------------------+------------------------------------------------------------
family_id | integer | not null default nextval('family_family_id_seq'::regclass)
familyserialno | integer | not null
sourcetype | character varying(20) | not null
familyname | character varying(100) |
familynamelocallang | character varying(255) |
hofname | character varying(100) | not null
hofnamelocallang | character varying(255) | not null
targetedfamily | boolean |
homeless | boolean |
landless | boolean |
dependentonlabour | boolean |
womenprimaryearner | boolean |
landlinenumber | character varying(20) |
username | character varying(20) | not null
adddate | timestamp without time zone | not null default now()
updatedate | timestamp without time zone | not null default now()
node_id | integer | not null
Indexes:
"PK_family" PRIMARY KEY, btree (family_id)
"family_idx" UNIQUE, btree (familyserialno, node_id)
Foreign-key constraints:
"family_fk" FOREIGN KEY (node_id) REFERENCES hierarchynode_master(node_id)
Referenced by:
TABLE "agriland" CONSTRAINT "FK_agriland_family" FOREIGN KEY (family_id) REFERENCES family(family_id) ON UPDATE RESTRICT ON DELETE RESTRICT
TABLE "currentloans" CONSTRAINT "FK_currentloans_family" FOREIGN KEY (family_id) REFERENCES family(family_id) ON UPDATE RESTRICT ON DELETE RESTRICT
TABLE "family_address" CONSTRAINT "FK_family_address_family" FOREIGN KEY (family_id) REFERENCES family(family_id) ON UPDATE RESTRICT ON DELETE RESTRICT
TABLE "family_basic_info" CONSTRAINT "FK_family_basic_info_family" FOREIGN KEY (family_id) REFERENCES family(family_id) ON UPDATE RESTRICT ON DELETE RESTRICT
TABLE "family_entitlement" CONSTRAINT "FK_family_entitlement_family" FOREIGN KEY (family_id) REFERENCES family(family_id) ON UPDATE RESTRICT ON DELETE RESTRICT
TABLE "livestock" CONSTRAINT "FK_livestock_family" FOREIGN KEY (family_id) REFERENCES family(family_id) ON UPDATE RESTRICT ON DELETE RESTRICT
TABLE "member" CONSTRAINT "FK_member_family" FOREIGN KEY (family_id) REFERENCES family(family_id) ON UPDATE RESTRICT ON DELETE RESTRICT
TABLE "otherassets" CONSTRAINT "FK_otherassets_family" FOREIGN KEY (family_id) REFERENCES family(family_id) ON UPDATE RESTRICT ON DELETE RESTRICT
Triggers:
tr_family_audit AFTER INSERT OR DELETE OR UPDATE ON family FOR EACH ROW EXECUTE PROCEDURE fn_family_audit()
tr_family_updatedate BEFORE UPDATE ON family FOR EACH ROW EXECUTE PROCEDURE fn_modify_updatedate_column()
nucleus4=#
Here's family_audit
nucleus4=# \d family_audit;
Table "public.family_audit"
Column | Type | Mod
---------------------+-----------------------------+----------------------------------
familyaudit_id | integer | not null default nextval('family_
operation | character(1) | not null
family_id | integer | not null
familyserialno | integer | not null
sourcetype | character varying(20) | not null
familyname | character varying(100) |
familynamelocallang | character varying(255) |
hofname | character varying(100) | not null
hofnamelocallang | character varying(255) | not null
targetedfamily | boolean |
homeless | boolean |
landless | boolean |
dependentonlabour | boolean |
womenprimaryearner | boolean |
landlinenumber | character varying(20) |
username | character varying(20) | not null
adddate | timestamp without time zone | not null default now()
node_id | integer | not null
Indexes:
"PK_family_audit" PRIMARY KEY, btree (familyaudit_id)
nucleus4=#
Here's the trigger
CREATE TRIGGER tr_family_audit
AFTER INSERT OR UPDATE OR DELETE ON public.family
FOR EACH ROW EXECUTE PROCEDURE fn_family_audit();
I would appreciate any hints.
Thank you,
BR,
~A
Your problem is here:
NEW.hofnamelocallang
Your insert has one extra column (apparently NEW.node_id). Try changing your insert to:
INSERT INTO public.family_audit values(
DEFAULT, 'I',NEW.family_id, NEW.familyserialno,
NEW.sourcetype, NEW.familyname,
NEW.familynamelocallang, NEW.hofname, NEW.hofnamelocallang,
NEW.targetedfamily, NEW.homeless,
NEW.landless, NEW.dependentonlabour, NEW.womenprimaryearner,
NEW.landlinenumber, NEW.username , now()
);
The error you are getting is basically saying that you were trying to insert NEW.hofnamelocallang into targetedfamily column (which is boolean, not varchar) because of the extra column you were putting in the insert sentence.
I would advice that, when you are performing an insert, for sanity reasons, always enumerate the columns you are putting values into. Something like this:
insert into table foo
(col1, col2, col3) -- column enumeration here
values
(1, 2, 3);