Postgres SQL Table Partitioning by Range Timestamp not Unique key Collision - postgresql

I have an issue when trying to modify and existing PostgreSQL (version 13.3) table to support partitioning it gets stuck when inserting the new data from the old table because the inserted timestamp in some cases may not be unique, so it fails on execution.
The partition forces me to create the primary to be the range (timestamp) value. You can see the new table definition below:
CREATE TABLE "UserFavorites_master" (
"Id" int4 NOT NULL GENERATED BY DEFAULT AS IDENTITY,
"UserId" int4 NOT NULL,
"CardId" int4 NOT NULL,
"CreationDate" timestamp NOT NULL,
CONSTRAINT "PK_UserFavorites_CreationDate" PRIMARY KEY ("CreationDate")
) partition by range ("CreationDate");
The original table didn't have a constraint on timestamp to either be unique or a primary key nor would we particularly want that but that seems to be a requirement of partitioning. Looking for alternatives or good ideas to solve the issue.
You can see the full code below:
alter table "UserFavorites" rename to "UserFavorites_old";
CREATE TABLE "UserFavorites_master" (
"Id" int4 NOT NULL GENERATED BY DEFAULT AS IDENTITY,
"UserId" int4 NOT NULL,
"CardId" int4 NOT NULL,
"CreationDate" timestamp NOT NULL,
CONSTRAINT "PK_UserFavorites_CreationDate" PRIMARY KEY ("CreationDate")
) partition by range ("CreationDate");
-- Frome Reference: https://stackoverflow.com/a/53600145/1190540
create or replace function createPartitionIfNotExists(forDate timestamp) returns void
as $body$
declare yearStart date := date_trunc('year', forDate);
declare yearEndExclusive date := yearStart + interval '1 year';
declare tableName text := 'UserFavorites_Partition_' || to_char(forDate, 'YYYY');
begin
if to_regclass(tableName) is null then
execute format('create table %I partition of "UserFavorites_master" for values from (%L) to (%L)', tableName, yearStart, yearEndExclusive);
-- Unfortunatelly Postgres forces us to define index for each table individually:
--execute format('create unique index on %I (%I)', tableName, 'UserId'::text);
end if;
end;
$body$ language plpgsql;
do
$$
declare rec record;
begin
loop
for rec in 2015..2030 loop
-- ... and create a partition for them
perform createPartitionIfNotExists(to_date(rec::varchar,'yyyy'));
end loop;
end
$$;
create or replace view "UserFavorites" as select * from "UserFavorites_master";
insert into "UserFavorites" ("Id", "UserId", "CardId", "CreationDate") select * from "UserFavorites_old";
It fails on the Last line with the following error:
SQL Error [23505]: ERROR: duplicate key value violates unique constraint "UserFavorites_Partition_2020_pkey"
Detail: Key ("CreationDate")=(2020-11-02 09:38:54.997) already exists.
ERROR: duplicate key value violates unique constraint "UserFavorites_Partition_2020_pkey"
Detail: Key ("CreationDate")=(2020-11-02 09:38:54.997) already exists.
ERROR: duplicate key value violates unique constraint "UserFavorites_Partition_2020_pkey"
Detail: Key ("CreationDate")=(2020-11-02 09:38:54.997) already exists.

No, partitioning doesn't force you to create a primary key. Just omit that line, and your example should work.
However, you definitely always should have a primary key on your tables. Otherwise, you can end up with identical rows, which is a major headache in a relational database. You might have to clean up your data.

#Laurenz Albe is correct, it seems I also have the ability to specify multiple keys though it may affect performance as referenced here Multiple Keys Performance, even indexing the creation date of the partition seemed to make the performance worse.
You can see a reference to multiple keys below, you mileage may vary.
CREATE TABLE "UserFavorites_master" (
"Id" int4 NOT NULL GENERATED BY DEFAULT AS IDENTITY,
"UserId" int4 NOT NULL,
"CardId" int4 NOT NULL,
"CreationDate" timestamp NOT NULL,
CONSTRAINT "PK_UserFavorites" PRIMARY KEY ("Id", "CreationDate")
) partition by range ("CreationDate");

Related

PGSQL Insert into A get ID to Insert into B

Table A is a kind of unique sequence for all my tables.
-- Table: public."IdCentral"
-- DROP TABLE public."IdCentral";
CREATE TABLE public."IdCentral"
(
"Id" bigint NOT NULL DEFAULT nextval('"IdCentral_Id_seq"'::regclass),
"Tag" character varying(127) COLLATE pg_catalog."default",
CONSTRAINT "IdCentral_pkey" PRIMARY KEY ("Id")
)
WITH (
OIDS = FALSE
)
TABLESPACE pg_default;
Table B is any table of my database
-- Table: public."Users"
-- DROP TABLE public."Users";
CREATE TABLE public."Users"
(
"Id" bigint NOT NULL,
"Login" character varying(30) COLLATE pg_catalog."default" NOT NULL,
CONSTRAINT "Users_pkey" PRIMARY KEY ("Id"),
CONSTRAINT "PK" FOREIGN KEY ("Id")
REFERENCES public."IdCentral" ("Id") MATCH FULL
ON UPDATE NO ACTION
ON DELETE NO ACTION
)
WITH (
OIDS = FALSE
)
TABLESPACE pg_default;
ALTER TABLE public."Users"
OWNER to dba;
When I want to insert into B, I need to create a new record in A passing the B Table name as Tag.
What you want is CURRVAL:
SELECT CURRVAL('IdCentral_Id_seq');
... which will give you the current value for the ID sequence after insert. For safety, it's best to use it inside a transaction, especially if you're combining it with load-balancing:
BEGIN;
INSERT INTO "a" ...
INSERT INTO "b" VALUES ( CURRVAL('IdCentral_Id_seq', ... )
COMMIT;
That being said, it appears that you're implementing a "universal ID" system for your database. This is something every new DBA tries (I did), and it's inevitably a bad idea which you end up spending a lot of time refactoring out later. If there's some reason you genuinely need some kind of universal ID, consider using a UUID instead.

Access database, Sql query , Error "Syntax error in DROP TABLE or DROP INDEX."

This is the query , running this in C#.
n getting above error
"DROP TABLE IF EXISTS `NATIONAL_ID_ISSUANCE_CENTER`;
CREATE TABLE `NATIONAL_ID_ISSUANCE_CENTER` (
`ID` INTEGER NOT NULL AUTO_INCREMENT,
`NAME` VARCHAR(100),
`APPLICATION_ID` INTEGER,
`STATUS` INTEGER,
`CREATED_BY` INTEGER,
`UPDATED_BY` INTEGER,
`CREATED_DATE` DATETIME,
`UPDATED_DATE` DATETIME,
`THIRD_PARTY_ID` INTEGER,
`PROVINCE_ID` INTEGER,
INDEX (`APPLICATION_ID`),
PRIMARY KEY (`ID`),
INDEX (`PROVINCE_ID`),
INDEX (`THIRD_PARTY_ID`)
)"
You can't put an IF statement inside Drop and Create statements. Anytime you want to drop a table that you're not sure exists, use the following:
IF(OBJECT_ID('[Database].[Schema].[TableName]') is not null)
BEGIN
DROP TABLE [Database].[Schema].[TableName];
END;
Please note you should replace [Database], [Schema], and [TableName] with the appropriate database, schema, and table names, respectively.

Trigger after insert, update a related model Postgresql

This is the first time I'm creating a trigger, so I've got a little bit confused. I'm following this guide.
Here is what I've done so far:
DROP TRIGGER IF EXISTS "update_metas" ON "post";
CREATE TRIGGER "update_metas"
AFTER INSERT ON "post"
FOR EACH ROW EXECUTE PROCEDURE update_post_count();
I have two tables: user and post. What I need to do is to increment the column user.postCount for each new post created. The foreign key is post.user_id
The procedure I'm creating is the following:
CREATE FUNCTION update_post_count() RETURNS TRIGGER AS $updates_user_postCount$
BEGIN
-- I know that NEW contains the new post info, so I
-- can gather the user_id by doing NEW.post_id.
--
-- What exactly should I do here?
RETURN NEW;
END;
$updates_user_postCount$ LANGUAGE plpgsql;
How should I structure this procedure? Can I just use a direct SQL query, something like:
UPDATE "user"
SET "user"."post_count" = "user"."post_count" + 1
WHERE "user"."_id" = NEW.idol_id;
UPDATE
I've tried using that SQL statement inside the procedure, but it returns the error error: column "user" of relation "user" does not exist.
Here is the SQL statement that I used to create both user and post tables:
CREATE TABLE IF NOT EXISTS "user" (
_id BIGSERIAL UNIQUE,
__id TEXT UNIQUE,
fbid VARCHAR(100),
name VARCHAR(100) NOT NULL,
email VARCHAR(100) NOT NULL UNIQUE,
password VARCHAR(512) NOT NULL,
profile_picture VARCHAR(512),
profile_cover VARCHAR(512),
profile_about TEXT,
profile_birth_date BIGINT,
social_facebook VARCHAR(256),
social_twitter VARCHAR(256),
social_instagram VARCHAR(256),
post_count BIGINT,
highlighted BOOLEAN,
idol BOOLEAN,
free BOOLEAN,
blocked BOOLEAN
);
CREATE TABLE IF NOT EXISTS "post" (
_id BIGSERIAL UNIQUE,
__id TEXT UNIQUE,
idol_id BIGINT,
removed BOOLEAN,
free BOOLEAN,
created_at BIGINT,
hashtags VARCHAR(1024),
audio_src VARCHAR(512),
audio_size INTEGER,
audio_length INTEGER,
FOREIGN KEY ("idol_id") REFERENCES "user"("_id")
);
Your trigger function is largely correct. The only problem is that an UPDATE statement cannot use the table.column notation.
From the documentation: Do not include the table's name in the specification of a target column — for example, UPDATE tab SET tab.col = 1 is invalid.
CREATE FUNCTION update_post_count() RETURNS TRIGGER AS $updates_user_postCount$
BEGIN
UPDATE "user"
SET "post_count" = "post_count" + 1
WHERE "_id" = NEW.idol_id;
RETURN NEW;
END;
$updates_user_postCount$ LANGUAGE plpgsql;

Postgres before insert trigger using sequence from another table

Using Postgres, what I would like to achieve is to be able to have many different instrument types, with corresponding [TYPE].instrument tables, which all have a unique ID in the table, but also reference a unique ID in the instrument.master table. I have the following:
create schema instrument
CREATE TABLE instrument.type (
id smallserial NOT NULL,
name text not null,
code text not null,
CONSTRAINT pk_instrument_type PRIMARY KEY (id)
);
ALTER TABLE instrument.type ADD CONSTRAINT unq_instrument_type_code UNIQUE(code);
ALTER TABLE instrument.type ADD CONSTRAINT unq_instrument_type_name UNIQUE(name);
insert into instrument.type (name, code) values ('futures', 'f');
CREATE TABLE instrument.master (
id serial NOT NULL,
type smallint not null references instrument.type (id),
timestamp timestamp with time zone not null,
CONSTRAINT pk_instrument_master PRIMARY KEY (id)
);
CREATE TABLE futures.definition (
id smallserial NOT NULL,
code text not null,
CONSTRAINT pk_futures_definition PRIMARY KEY (id)
);
ALTER TABLE futures.definition ADD CONSTRAINT unq_futures_definition_code UNIQUE(code);
insert into futures.definition (code) values ('ED');
CREATE TABLE futures.instrument (
id smallserial NOT NULL,
master serial not null references instrument.master (id),
definition smallint not null references futures.definition (id),
month smallint not null,
year smallint not null,
CONSTRAINT pk_futures_instrument PRIMARY KEY (id),
check (month >= 1),
check (month <= 12),
check (year >= 1900)
);
ALTER TABLE futures.instrument ADD CONSTRAINT unq_futures_instrument UNIQUE(definition, month, year);
CREATE OR REPLACE FUNCTION trigger_master_futures()
RETURNS trigger AS
$BODY$
BEGIN
insert into instrument.master (type, timestamp)
select id, current_timestamp from instrument.type where code = 'f';
NEW.master := currval('instrument.master_id_seq');
RETURN NEW;
END;
$BODY$
LANGUAGE 'plpgsql' VOLATILE;
create trigger trg_futures_instrument before insert on futures.instrument
for each row
execute procedure trigger_master_futures();
I then test with:
insert into futures.instrument (definition, month, year)
select id, 3, 2015 from futures.definition where code = 'ED';
Everything works almost as I would like it to. The only issue is that somehow, instrument.master.id ends up being one more than futures.instrument.master. I am not sure what I need to do to achieve the behavior I want, which is that whenever an entry is inserted into futures.instrument, an entry should be inserted into instrument.master, and the id entry of the latter should be inserted into the master entry of the former. I actually think it should have failed since the foreign key relationship is violated somehow.
As it turns out, everything was correct. The issue was that in futures.instrument, the type of the master column is serial, and it should have been int.

Generate a random key including current inserting column value in oracle is it possible to create

I'm trying to generate a random key including current inserting column value in oracle is it possible to create?
CREATE TABLE MY_TABLE
(
KEY VARCHAR2(12) not null,
SITEID varchar2(25) not null,
SITENAME varchar2(50),
CONSTRAINT MY_pk PRIMARY KEY (KEY)
);
INSERT INTO MY_TABLE (KEY, SITEID, SITENAME)
VALUES(('ABCD001'||SITEID), 'HYD001', 'HYDERABADSITE');
It would be better to use a BEFORE TRIGGER to do this. like,
CREATE OR REPLACE
TRIGGER my_table_trigger
BEFORE INSERT ON my_table
FOR EACH ROW
BEGIN
:NEW.KEY := 'ABCD001'||:NEW.siteid;
END;