I'm just getting into Postgis and I'm running into an interesting problem:
I've added a unique constraint to my table between a varchar name column with a generic geography geom column:
CREATE TABLE public.locations
(
id uuid NOT NULL,
name character varying(255) COLLATE pg_catalog."default",
geom geography,
inserted_at timestamp(0) without time zone NOT NULL,
updated_at timestamp(0) without time zone NOT NULL,
CONSTRAINT locations_pkey PRIMARY KEY (id)
)
I've added in a unique constraint using btree
CREATE INDEX locations_geom_index
ON public.locations USING btree
(geom ASC NULLS LAST)
TABLESPACE pg_default;
-- Index: locations_name_geom_index
-- DROP INDEX public.locations_name_geom_index;
CREATE UNIQUE INDEX locations_name_geom_index
ON public.locations USING btree
(name COLLATE pg_catalog."default" ASC NULLS LAST, geom ASC NULLS LAST)
TABLESPACE pg_default;
It looks like the unique index is not being respected. I read online that I need to use a GIST index (but that won't allow unique values). How can I properly add a unique constraint so I can be sure that something with the same name and GPS location won't be duplicated?
Since I will be storing points, should I change this to be a geography(Point, 4326)?
Related
I have a simple table that we use to record debug logs.
There are only 1000 rows in the table.
There are a handful of columns - id (primary), date (indexed), level, a few other small fields and message which could be a large string.
if I query:
select id from public.log
the query completes very quickly (less than 1 second)
if I query:
select id,date from public.log
or
select * from public.log
it takes 1 minute and 28 seconds to complete!
90 Seconds to read 1000 records from a database!
however if I query:
select *
from public.log
where id in (select id from public.log)
it completes in about 1 second.
And here is the CREATE - I just had pgAdmin generate them for me
-- Table: public.inettklog
-- DROP TABLE public.inettklog;
CREATE TABLE public.inettklog
(
id integer NOT NULL DEFAULT nextval('inettklog_id_seq'::regclass),
date timestamp without time zone NOT NULL,
thread character varying(255) COLLATE pg_catalog."default" NOT NULL,
level character varying(20) COLLATE pg_catalog."default" NOT NULL,
logger character varying(255) COLLATE pg_catalog."default" NOT NULL,
customlevel integer,
source character varying(64) COLLATE pg_catalog."default",
logentry json,
CONSTRAINT inettklog_pkey PRIMARY KEY (id)
)
WITH (
OIDS = FALSE
)
TABLESPACE pg_default;
ALTER TABLE public.inettklog
OWNER to postgres;
-- Index: inettklog_ix_logdate
-- DROP INDEX public.inettklog_ix_logdate;
CREATE INDEX inettklog_ix_logdate
ON public.inettklog USING btree
(date)
TABLESPACE pg_default;
Your table is extremely bloated. Given your other question, this extreme bloat is not surprising.
You can fix this with a VACUUM FULL.
Going forward, you should avoid getting into this situation in the first place, by deleting records as they become obsolete rather than waiting until 99.998% of them are obsolete before acting.
I have trigger function that automatically creates child tables based on date column from parent table (table 1). Hovewer I have to make modification to do that based on date column from another table (table 2)!
Is this possible at all? I have foreign key in table 1 which is linked with a id column in table 2.
I searched over the internet but mostly found different scripts for task I already solved (date column in parent table, not in another table).
EXAMPLE: Make monthly partitions of table invoice_details based on invoice_date in table invoice (foreign key invoice_details.invoice_id - > invoice.invoice_id)
CREATE TABLE public.invoice_details
(
id integer NOT NULL,
invoice_id integer NOT NULL,
charge_type integer,
charge_amount numeric(15,5),
charge_tax numeric(15,5),
charge_status character varying COLLATE pg_catalog."default")
TABLESPACE pg_default;
CREATE TABLE public.invoice
(
invoice_id integer NOT NULL,
customer character varying COLLATE pg_catalog."default",
invoice_date date NOT NULL)
I created two tables using Django models and the scripts look something like this
I am using PostgreSQL 10
production table:
CREATE TABLE public.foods_food(
id integer NOT NULL DEFAULT nextval('foods_food_id_seq'::regclass),
code character varying(100) COLLATE pg_catalog."default",
product_name character varying(255) COLLATE pg_catalog."default",
brands character varying(255) COLLATE pg_catalog."default",
quantity character varying(255) COLLATE pg_catalog."default",
last_modified_datetime timestamp with time zone NOT NULL,
created_at timestamp with time zone NOT NULL
)
staging table:
CREATE TABLE public.foods_temp(
id integer NOT NULL DEFAULT nextval('foods_temp_id_seq'::regclass),
code character varying(100) COLLATE pg_catalog."default",
product_name character varying(255) COLLATE pg_catalog."default",
)
I copied a CSV file to the staging table and than I tried to copy the columns from the staging table to the production table using the following query.
INSERT INTO foods_food
SELECT * FROM foods_temp;
But I got this error.
ERROR: null value in column "created_at" violates not-null constraint
I can set the created_at column to accept null in order to make it work but I want the created_at values to be auto populated when entries are inserted.
Is there other way to copy columns to the production table and automatically insert the timestamp?
Then you need to set default values:
ALTER TABLE public.foods_food ALTER last_modified_datetime
SET DEFAULT current_timestamp;
ALTER TABLE public.foods_food ALTER created_at
SET DEFAULT current_timestamp;
I am using Postgresql with citus extension for sharding and unable to shard tables like below.
Below table has a primary key and 2 unique keys. I am trying to shard against column with primary key i.e pid.
Note: I am not allowed to change the table structure. These tables are created by tool.
CREATE TABLE person
(
pid bigint NOT NULL,
name character varying(100),
address_pid bigint NOT NULL,
address_type character varying(100),
CONSTRAINT id_pkey PRIMARY KEY (pid),
CONSTRAINT addr_id UNIQUE (address_pid),
CONSTRAINT addr_type_id UNIQUE (address_type, address_pid)
);
This my sharding query:
select create_distributed_table('person', 'pid');
Error it throw is:
Error: Distributed relations cannot have UNIQUE, EXCLUDE, or PRIMARY KEY constraints that do not include the partition column
Can anyone help me with sharding these kind of tables?
#CraigKerstiens Addition to this question:
How to handle sharding when we have multiple foreign keys like this one.
CREATE TABLE table
(
pid bigint NOT NULL,
search_order integer NOT NULL,
resource_pid bigint NOT NULL,
search_pid bigint NOT NULL,
CONSTRAINT hfj_search_result_pkey PRIMARY KEY (pid),
CONSTRAINT idx_searchres_order UNIQUE (search_pid, search_order),
CONSTRAINT fk_searchres_res FOREIGN KEY (resource_pid)
REFERENCES public.table1 (res_id) MATCH SIMPLE
ON UPDATE NO ACTION
ON DELETE NO ACTION,
CONSTRAINT fk_searchres_search FOREIGN KEY (search_pid)
REFERENCES public.table2 (pid) MATCH SIMPLE
ON UPDATE NO ACTION
ON DELETE NO ACTION
)
Assuming that table1 and table2 are already sharded.
Within Citus at this time you cannot have a unique constraint that doesn't include they column you are partitioning on. In this case, it'd be possible to enforce addresses were unique to the person id, but not globally unique. To do that you could:
CREATE TABLE person
(
pid bigint NOT NULL,
name character varying(100),
address_pid bigint NOT NULL,
address_type character varying(100),
CONSTRAINT id_pkey PRIMARY KEY (pid),
CONSTRAINT addr_id UNIQUE (pid, address_pid),
CONSTRAINT addr_type_id UNIQUE (pid, address_type, address_pid)
);
On a quite simple table
CREATE TABLE collectionmaster
(
id character varying(32) NOT NULL,
version integer,
publisherid character varying(12),
title character varying(1024),
subtitle character varying(1024),
type character varying(255) NOT NULL,
CONSTRAINT collectionmaster_pkey PRIMARY KEY (id)
)
WITH (
OIDS=FALSE
);
CREATE INDEX idx_coll_title
ON collectionmaster
USING btree
(title COLLATE pg_catalog."default");
I tried to add a unique check either via unique index
CREATE UNIQUE INDEX idx_uni_coll_title
ON collectionmaster
USING btree
(publisherid COLLATE pg_catalog."default", title COLLATE pg_catalog."default", (COALESCE(subtitle, 'no_sub'::character varying)) COLLATE pg_catalog."default");
or via unique constraint
ALTER TABLE collectionmaster
ADD CONSTRAINT uni_coll_title UNIQUE(publisherid, title, subtitle);
to intercept accidentally multiple creation by a java service (spring-jpa on hibernate) executed in several threads. But strangely neither of index and constraint works as expected. Eight records are added by eight threads which are not unique according to either constraint or index in separate transaction, but no unique violations are thrown, and all records are inserted into the table. And none of the values are null, as was the problem in other questions here.
The constaint was not deferred, index as constraint where valid.
Postgres version is 9.2.
I am quite clueless here.
EDIT: These are the results of a query in this table, extracted from pgAdmin (hard to format it nicer here):
"id";"version";"publisherid";"title";"subtitle";"type"
"53b3df625baf40bf885b48daa366fbc8";1;"5109138";"Titel";"Untertitel";"set"
"2673ef9a33f84289995d6f7288f07b46";1;"5109138";"Titel";"Untertitel";"set"
"ef7c385205034fdc89fe39e3eca48408";1;"5109138";"Titel";"Untertitel";"set"
"527922f2f3464f91826dbae2e2b67caf";1;"5109138";"Titel";"Untertitel";"set"
"794638a725324319852d10b828257df7";1;"5109138";"Titel";"Untertitel";"set"
"dbe2201058974d63a2107131f0080233";1;"5109138";"Titel";"Untertitel";"set"
"cbb77c7c1adb415db006853a6f6244ac";1;"5109138";"Titel";"Untertitel";"set"
"0b6606fe015040fbbc85444361ab414c";1;"5109138";"Titel";"Untertitel";"set"
Even on these I can execute
insert into collectionmaster(id, version, publisherid, title, subtitle, type) values('aaa',1,'5109138','Titel','Untertitel','set')
without getting a constraint violation. I can't believe it, too, but this is the problem...
Make sure your spring-jpa config spring.jpa.hibernate.ddl-auto is not set to create-drop.
That way, hibernate will always drop & re-create your whole schema, which won't contain your custom unique constraints, nor your indexes.