PostgreSQL tables creation with wrong order - postgresql

I have an .sql file, that creates lots of tables, that are related to each other.
I made other file for testing, that holds only two statements:
CREATE TABLE "USER" (
"id" bigint NOT NULL,
"name" varchar(50),
PRIMARY KEY ("id"));
CREATE TABLE "PERSON" (
"id" bigint NOT NULL,
"name" varchar(50),
"user" bigint,
PRIMARY KEY ("id"),
CONSTRAINT "fk_user" FOREIGN KEY ("user") REFERENCES "USER" ("id"));
This works fine if i'm trying to execute such file, but if i have other order - where table "PERSON" is created first - i'm getting ERROR: relation "USER" does not exist.
Is it possible to make some changes (or use some additional options when running 'psql' command), leaving the order as it is, to make it work?
EDIT: I understand why this error happens in given case, but i was thinking about some solution, where i don't need to change the order of my CREATE statements (Imagine you have hundreds of tables)... In MySQL you can simply use SET FOREIGN_KEY_CHECKS=0; and this will work. Do i have similar possibilities in PostgreSQL?

If you want table a to reference table b, you must either create table b before table a, or add the foreign key reference after creation:
ALTER TABLE a ADD FOREIGN KEY (a_col) REFERENCES b(b_col);
That works to create two tables that reference each other, too, but you won't be able to create rows unless you make one of them DEFERRABLE INITIALLY DEFERRED.

You are getting the error because at the point you are creating the foreign key on the the PERSON table it references the USER table which does not exist yet.
You can work round this issue by separating out FOREIGN KEY CONSTRAINT into it's own statement and applying this after you have created both tables:
CREATE TABLE "PERSON" (
"id" bigint NOT NULL,
"name" varchar(50),
"user" bigint,
PRIMARY KEY ("id"));
CREATE TABLE "USER" (
"id" bigint NOT NULL,
"name" varchar(50),
PRIMARY KEY ("id"));
ALTER TABLE "PERSON"
ADD CONSTRAINT fk_user
FOREIGN KEY ("user")
REFERENCES "USER" (id);

Related

Dropping unique constraint on foreign key reference of junction table

I'm creating a database in PostgreSQL and want to include a many-to-many relationship between the tables. The two tables I want to include are as follows:
CREATE TABLE "meter" (
"id" integer PRIMARY KEY,
"nmi" integer,
"next_scheduled_read_date" timestamp
);
CREATE TABLE "register" (
"id" text PRIMARY KEY,
"value" text
);
The many-to-many relationship I want to have is between meter id and register id. I have then created the junction table below:
CREATE TABLE "meter_registers" (
"meter_id" integer NOT NULL,
"register_id" text NOT NULL,
PRIMARY KEY ("meter_id", "register_id"),
FOREIGN KEY ("meter_id") REFERENCES "meter" ("id") ON UPDATE CASCADE,
FOREIGN KEY ("register_id") REFERENCES "register" ("id") ON UPDATE CASCADE
);
I then want to create a table that references the meter_id and register_id values from the junction table above which is structured as follows:
CREATE TABLE "demand_data" (
"upload_id" integer PRIMARY KEY,
"nmi" integer,
"meter" integer,
"register" text,
"start" timestamp,
"end" timestamp,
"duration" Time,
"demand" double precision
);
Where the meter and register reference the corresponding junction table columns. However, as the values of the junction table will not be unique I can't simply add a foreign key for the meter and register columns so I run into an error when I run the following:
ALTER TABLE "demand_data" ADD FOREIGN KEY ("meter") REFERENCES "meter_registers" ("meter_id");
ALTER TABLE "demand_data" ADD FOREIGN KEY ("register") REFERENCES "meter_registers" ("register_id");
ERROR: there is no unique constraint matching given keys for referenced table "meter_registers"
Is there a way to possibly reference the junction table columns in the demand_data table without the foreign key constraint? I know it's possible to do with a separate query once some data has been added using inner joins however, is it possible to do it through database table set up?
Yes. A compound foreign key.
ALTER TABLE "demand_data"
ADD FOREIGN KEY ("meter","register")
REFERENCES "meter_registers"( "meter_id", "register_id");
Note: Not directly related you should avoid those dreaded double quotes.
If demand_data references meter_registers, it should reference its primary key. So add a single foreign key constraint on both columns.
If you want two separate foreign keys,you should probably reference meter and register directly.

Citus: How can I add self referencing table in distributed tables list

I'm trying to run create_distributed_table for tables which i need to shard and almost all of the tables have self relation ( parent child )
but when I run SELECT create_distributed_table('table-name','id');
it throws error cannot create foreign key constraint
simple steps to reproduce
CREATE TABLE TEST (
ID TEXT NOT NULL,
NAME CHARACTER VARYING(255) NOT NULL,
PARENT_ID TEXT
);
ALTER TABLE TEST ADD CONSTRAINT TEST_PK PRIMARY KEY (ID);
ALTER TABLE TEST ADD CONSTRAINT TEST_PARENT_FK FOREIGN KEY (PARENT_ID) REFERENCES TEST (ID);
ERROR
citus=> SELECT create_distributed_table('test','id');
ERROR: cannot create foreign key constraint
DETAIL: Foreign keys are supported in two cases, either in between two colocated tables including partition column in the same ordinal in the both tables or from distributed to reference tables
For the time being, it is not possible to shard a table on PostgreSQL without dropping the self referencing foreign key constraints, or altering them to include a separate and new distribution column.
Citus places records into shards based on the hash values of the distribution column values. It is most likely the case that the hashes of parent and child id values are different and hence the records should be stored in different shards, and possibly on different worker nodes. PostgreSQL does not have a mechanism to create foreign key constraints that reference records on different PostgreSQL clusters.
Consider adding a new column tenant_id and adding this column to the primary key and foreign key constraints.
CREATE TABLE TEST (
tenant_id INT NOT NULL,
id TEXT NOT NULL,
name CHARACTER VARYING(255) NOT NULL,
parent_id TEXT NOT NULL,
FOREIGN KEY (tenant_id, parent_id) REFERENCES test(tenant_id, id),
PRIMARY KEY (tenant_id, id)
);
SELECT create_distributed_table('test','tenant_id');
Note that parent and child should always be in the same tenant for this to work.

How to use timescale hypertables with foreign keys and keep a one-to-many relation?

I am trying to create a database with minimum redundancy in mind. We would like to use the timescaledb hypertables (I run postgreSQL v. 12 and timescaledb v. 1.7.4). The postgreSQL code to create the tables are as follows - you can see the dbdiagram here https://dbdiagram.io/d/5f992f0e3a78976d7b797ca2 or view the tables here Image of database
CREATE TABLE "datapoints" (
"id" bigserial UNIQUE NOT NULL,
"tstz" timestamptz NOT NULL,
"entity_id" bigint NOT NULL,
"value" real NOT NULL,
PRIMARY KEY ("tstz", "entity_id")
);
CREATE TABLE "datapoint_quality" (
"tstz" timestamptz NOT NULL,
"datapoint_id" bigint NOT NULL,
"flag_id" bigint NOT NULL,
PRIMARY KEY ("tstz", "datapoint_id", "flag_id")
);
CREATE TABLE "quality_flags" (
"id" bigserial PRIMARY KEY,
"value" text
);
CREATE TABLE "sensor_types" (
"id" bigserial PRIMARY KEY,
"name" text UNIQUE NOT NULL
);
CREATE TABLE "sensors" (
"tstz" timestamptz NOT NULL DEFAULT (now()),
"id" bigserial UNIQUE NOT NULL,
"name" text NOT NULL,
"parent" bigint NOT NULL,
"type" bigint NOT NULL,
PRIMARY KEY ("tstz", "id")
);
CREATE TABLE "datapoint_annotation" (
"tstz" timestamptz NOT NULL,
"datapoint_id" bigint NOT NULL,
"annotation_id" bigint NOT NULL,
PRIMARY KEY ("tstz", "datapoint_id", "annotation_id")
);
CREATE TABLE "annotations" (
"id" bigserial PRIMARY KEY NOT NULL,
"value" text NOT NULL
);
ALTER TABLE "datapoints" ADD FOREIGN KEY ("entity_id") REFERENCES "sensors" ("id");
ALTER TABLE "datapoint_quality" ADD FOREIGN KEY ("datapoint_id") REFERENCES "datapoints" ("id");
ALTER TABLE "datapoint_quality" ADD FOREIGN KEY ("flag_id") REFERENCES "quality_flags" ("id");
ALTER TABLE "sensors" ADD FOREIGN KEY ("parent") REFERENCES "sensors" ("id");
ALTER TABLE "sensors" ADD FOREIGN KEY ("type") REFERENCES "sensor_types" ("id");
ALTER TABLE "datapoint_annotation" ADD FOREIGN KEY ("datapoint_id") REFERENCES "datapoints" ("id");
ALTER TABLE "datapoint_annotation" ADD FOREIGN KEY ("annotation_id") REFERENCES "annotations" ("id");
CREATE UNIQUE INDEX ON "quality_flags" ("value");
CREATE UNIQUE INDEX ON "annotations" ("value");
So far so good - next I want to create the hypertables, which I do as:
SELECT create_hypertable('datapoint_annotation', 'tstz');
SELECT create_hypertable('datapoint_quality', 'tstz');
SELECT create_hypertable('datapoints', 'tstz');
SELECT create_hypertable('sensors', 'tstz');
This works well for the first two lines, but for the latter two I get the following error:
ERROR: cannot create a unique index without the column "tstz" (used in partitioning)
SQL state: TS103
I can include the tstz in the primary key as ("id", "tstz") and use that as foreign key, but this gives me a one-to-one relation, and for minimum redundancy I would like to have a one-to-many relation.
I am sure there should be some way to do this - so what am I missing?
I'll take the foreign key constraint from datapoint_quality to datapoints as an example.
To make that work with a partitioned table, you need a unique constraint on datapoint. As the error message tell you, such a constraint must contain the partitioning key. So you end up with
ALTER TABLE datapoints ADD UNIQUE (id, tstz);
To reference that unique constraint from datapoint_quality, you need to have the timestamp there too:
ALTER TABLE datapoint_quality ADD datapoints_tstz timestamp with time zone;
You have to fill it with the appropriate values:
UPDATE datapoint_quality AS dq
SET datapoints_tstz = d.tstz
FROM datapoints AS d
WHERE d.id = dq.datapoint_id;
Then set it NOT NULL:
ALTER TABLE datapoint_quality ALTER datapoints_tstz SET NOT NULL;
Now you can define your foreign key:
ALTER TABLE datapoint_quality
ADD FOREIGN KEY (datapoint_id, datapoints_tstz)
REFERENCES datapoints (id, tstz) MATCH FULL;
There is no other way to have foreign key constraints with partitioned tables.
After testing the proposed solution by Laurenz in a database I have and also after replicating the original database of this case. I use PostgreSQL 12.6 and timescaledb 1.7.5.
Basically, I arrived well until defining the Foreign Key for Table datapoint_quality:
ALTER TABLE datapoint_quality
ADD FOREIGN KEY (datapoint_id, datapoints_tstz)
REFERENCES datapoints (id, tstz) MATCH FULL;
The next error is present in both databases I've tested after several attempts (included above one) to define the foreign key to a hypertable:
ERROR: foreign keys to hypertables are not supported Blockquote SQL state: 0A000
According to https://docs.timescale.com/timescaledb/latest/overview/limitations/##distributed-hypertable-limitations, it looks like the above error is part of the hypertable limitations:
Foreign key constraints referencing a hypertable are not supported.
Considering this, does anyone know any solution at the DB level to establish the relationships (1..* or ...) among a table without hypertables to other tables with hypertables behind?
Maybe could be a solution to deal with this at even a REST API level (e.g. Django or Flask) given at timescaledb or PostgreSQL I have not found much more solutions.

Make a previously existing foreign key column have a unique constraint in postgres

I need to create a migration for an already existing table to make it's foreign key have a UNIQUE constraint. How do I do this?
From the examples I found in the documentation, it is mostly done when the table is created. The issue is I need to add this onto a column that already exists and is already set as a foreign key. This is what the table looks like at it's creation:
CREATE TABLE IF NOT EXISTS "myTable" (
"_id" SERIAL NOT NULL,
"myForeignKeyId" INTEGER NOT NULL,
"name" VARCHAR(255) NOT NULL,
CONSTRAINT "pk_myTable" PRIMARY KEY ("_id"),
CONSTRAINT "fk_myTable_myForeignKeyId" FOREIGN KEY ("myForeignKeyId") REFERENCES "myOtherTable" ("_id")
);
What I want to do is on a migration make myForeignKeyId unique. How do I do that?
I have tried to following:
CREATE UNIQUE INDEX CONCURRENTLY "myTable_myForeignKeyId"
ON province ("myForeignKeyId");
ALTER TABLE IF EXISTS "myTable"
ADD CONSTRAINT "myForeignKeyId"
UNIQUE USING INDEX "myTable_myForeignKeyId";
First off, when I try this in a migration I get the error:
CREATE INDEX CONCURRENTLY cannot run inside a transaction block
So that part cannot be done, but even just doing it through SQL, the second part doesn't work either as it claims myForeignKeyId already exists. Even if I add an ALTER COLUMN myForeignKeyId it just says there is an error on that line.
This seems like it should be a simple enough operation, how can I do this?
After digging some more found quite a simple way to do this, was clearly originally off target.
To add a unique constraint to a column:
ALTER TABLE "myTable"
ADD CONSTRAINT "myUniqueKeyNameOfChoice" UNIQUE ("myColumn");
To remove it:
ALTER TABLE "myTable"
DROP CONSTRAINT "myUniqueKeyNameOfChoice";

How do I create a check to make sure a value exists in another table?

Right now I have two tables, one that contains a compound primary key and another that that references one of the values of the primary key but is a one-to-many relationship between Product and Mapping. The following is an idea of the setup:
CREATE TABLE dev."Product"
(
"Id" serial NOT NULL,
"ShortCode" character(6),
CONSTRAINT "ProductPK" PRIMARY KEY ("Id")
)
CREATE TABLE dev."Mapping"
(
"LookupId" integer NOT NULL,
"ShortCode" character(6) NOT NULL,
CONSTRAINT "MappingPK" PRIMARY KEY ("LookupId", "ShortCode")
)
Since the ShortCode is displayed to the user as a six character string I don't want to have a another table to have a proper foreign key reference but trying to create one with the current design is not allowed by PostgreSQL. As such, how can I create a check so that the short code in the Mapping table is checked to make sure it exists?
Depending on the fine print of your requirements and your version of Postgres I would suggest a TRIGGER or a NOT VALID CHECK constraint.
We have just discussed the matter in depth in this related question on dba.SE:
Disable all constraints and table checks while restoring a dump
If I understand you correctly, you need a UNIQUE constraint on "Product"."ShortCode". Surely it should be declared NOT NULL, too.
CREATE TABLE dev."Product"
(
"Id" serial NOT NULL,
"ShortCode" character(6) NOT NULL UNIQUE,
CONSTRAINT "ProductPK" PRIMARY KEY ("Id")
);
CREATE TABLE dev."Mapping"
(
"LookupId" integer NOT NULL,
"ShortCode" character(6) NOT NULL REFERENCES dev."Product" ("ShortCode"),
CONSTRAINT "MappingPK" PRIMARY KEY ("LookupId", "ShortCode")
);
Your original "Product" table will allow this INSERT statement to succeed, but it shouldn't.
insert into dev."Product" ("ShortCode") values
(NULL), (NULL), ('ABC'), ('ABC'), ('ABC');
Data like that is just about useless.
select * from dev."Product"
id ShortCode
--
1
2
3 ABC
4 ABC
5 ABC