The problem is that when I want to convert a simple Postgresql table to timescaledb table or hypertable using created_at table field for indexing then it will show this error. The table name is orders. Here cas_admin_db_new is the databse name.
I have tried all the possible way. which is bellow but the orders table doesn't convert into hypertable.
SELECT create_hypertable('orders','created_at', chunk_time_interval => 6040800000000);
ERROR: cannot create a unique index without the column "created_at" (used in partitioning)
SELECT create_hypertable('public.orders','created_at', chunk_time_interval => 6040800000000);
ERROR: cannot create a unique index without the column "created_at" (used in partitioning)
cas_admin_db_new=# SELECT create_hypertable('public.orders','created_at', chunk_time_interval => 6040800000000, created_default_indexes=>FALSE);
ERROR: function create_hypertable(unknown, unknown, chunk_time_interval => bigint, created_default_indexes => boolean) does not exist
cas_admin_db_new=# SELECT create_hypertable('"ORDER"','created_at', chunk_time_interval => 6040800000000);
ERROR: relation "ORDER" does not exist
LINE 1: SELECT create_hypertable('"ORDER"','created_at', chunk_time_...
Timescale person here. The issue is that your schema probably lists some other column as a primary key (or UNIQUE index).
TimescaleDB requires that any PK/unique index includes all partitioning keys, in your case, created_at.
That's because we do this heavy underlying partitioning, and don't want to build global lookup structures to ensure uniqueness outside of what we already use for partitioning.
More info:
https://docs.timescale.com/timescaledb/latest/how-to-guides/schema-management/indexing/##best-practices
You need to drop your current primary key on table and create new composite primary key like so:
ALTER TABLE table_name ADD PRIMARY KEY (id, created_at);
But there is problem: Unfortunately ActiveRecord doesn't support composite primary key.
Related
I am recreating an existing table as a partitioned table in PostgreSQL 11.
After some research, I am approaching it using the following procedure so this can be done online while writes are still happening on the table:
add a check constraint on the existing table, first as not valid and then validating
drop the existing primary key
rename the existing table
create the partitioned table under the prior table name
attach the existing table as a partition to the new partitioned table
My expectation was that the last step would be relatively fast, but I don't really have a number for this. In my testing, it's taking about 30s. I wonder if my expectations are incorrect or if I'm doing something wrong with the constraint or anything else.
Here's a simplified version of the DDL.
First, the inserted_at column is declared like this:
inserted_at timestamp without time zone not null
I want to have an index on the ID even after I drop the PK for existing queries and writes, so I create an index:
create unique index concurrently my_events_temp_id_index on my_events (id);
The check constraint is created in one transaction:
alter table my_events add constraint my_events_2022_07_events_check
check (inserted_at >= '2018-01-01' and inserted_at < '2022-08-01')
not valid;
In the next transaction, it's validated (and the validation is successful):
alter table my_events validate constraint my_events_2022_07_events_check;
Then before creating the partitioned table, I drop the primary key of the existing table:
alter table my_events drop constraint my_events_pkey cascade;
Finally, in its own transaction, the partitioned table is created:
alter table my_events rename to my_events_2022_07;
create table my_events (
id uuid not null,
... other columns,
inserted_at timestamp without time zone not null,
primary key (id, inserted_at)
) partition by range (inserted_at);
alter table my_events attach partition my_events_2022_07
for values from ('2018-01-01') to ('2022-08-01');
That last transaction blocks inserts and takes about 30s for the 12M rows in my test database.
Edit
I wanted to add that in response to the attach I see this:
INFO: partition constraint for table "my_events_2022_07" is implied by existing constraints
That makes me think I'm doing this right.
The problem is not the check constraint, it is the primary key.
If you make the original unique index include both columns:
create unique index concurrently my_events_temp_id_index on my_events (id,inserted_at);
And if you make the new table have a unique index rather than a primary key on those two columns, then the attach is nearly instantaneous.
These seem to me like unneeded restrictions in PostgreSQL, both that the unique index on one column can't be used to imply uniqueness on the both columns, and that the unique index on both columns cannot be used to imply the primary key (nor even a unique constraint--but only a unique index).
This question already has an answer here:
ERROR: unique constraint on partitioned table must include all partitioning columns
(1 answer)
Closed last month.
I'm trying to create a partitioned table which refers to itself, creating a doubly-linked list.
CREATE TABLE test2 (
id serial NOT NULL,
category integer NOT NULL,
time timestamp(6) NOT NULL,
prev_event integer,
next_event integer
) PARTITION BY HASH (category);
Once I add primary key I get the following error.
alter table test2 add primary key (id);
ERROR: unique constraint on partitioned table must include all partitioning columns
DETAIL: PRIMARY KEY constraint on table "test2" lacks column "category" which is part of the partition key.
Why does the unique constrain require all partitioned columns to be included?
EDIT: Now I understand why this is needed: https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE-LIMITATIONS
Once I add PK with both columns it works.
alter table test2 add primary key (id, category);
But then adding the FK to itself doesn't work.
alter table test2 add foreign key (prev_event) references test2 (id) on update cascade on delete cascade;
ERROR: there is no unique constraint matching given keys for referenced table "test2"
Since PK is not just id but id-category I can't create FK pointing to id.
Is there any way to deal with this or am I missing something?
I would like to avoid using inheritance partitioning if possible.
EDIT2: It seems this is a known problem. https://www.reddit.com/r/PostgreSQL/comments/di5mbr/postgresql_12_foreign_keys_and_partitioned_tables/f3tsoop/
Seems that there is no straightforward solution. PostgreSQL simply doesn't support this as of v14. One solution is to use triggers to enforce 'foreign key' behavior. Other is to use multi-column foreign keys. Both are far from optimal.
How do you add a compound / composite index on a PostgreSQL table with TimescaleDB installed?
Following https://docs.timescale.com/latest/using-timescaledb/schema-management, you can add a compound / composite index to TimescaleDB by simply doing:
CREATE INDEX ON conditions (time DESC, cityid)
WHERE cityid IS NOT NULL;
time is a column with timestamps (The one used as primary key in TimescaleDB).
cityid is a column for a city identifier we might to often query for (As second parameter after the time series dates).
This can be done before or after converting the table to a hypertable.
To avoid bloating the index when the column cityid is often NULL, the statement WHERE cityid IS NOT NULL is for. Use this per default unless you are often searching for missing data (cityid IS NULL).
HeidiSQL generated the following creation code:
CREATE TABLE "books" (
"id" BIGINT NOT NULL,
"creation_date" TIMESTAMP NOT NULL,
"symbol" VARCHAR NOT NULL,
PRIMARY KEY ("id"),
UNIQUE INDEX "symbol" ("symbol")
)
;
COMMENT ON COLUMN "books"."id" IS E'';
COMMENT ON COLUMN "books"."creation_date" IS E'';
COMMENT ON COLUMN "books"."symbol" IS E'';
And when I try to submit, I get the following error:
Is it an HeidiSQL bug with PostgreSQL?
There is a recommendation note for how you should create an UNIQUE INDEX in PostgreSQL:
The preferred way to add a unique constraint to a table is ALTER TABLE ... ADD CONSTRAINT. The use of indexes to enforce unique constraints could be considered an implementation detail that should not be accessed directly. One should, however, be aware that there's no need to manually create indexes on unique columns; doing so would just duplicate the automatically-created index.
There are a couple of ways of creating a UNIQUE INDEX on PostgreSQL.
The first way is, as mentioned above, using ALTER TABLE on a previously created table:
ALTER TABLE books ADD UNIQUE ("symbol");
OR
ALTER TABLE books ADD CONSTRAINT UQ_SYMBOL UNIQUE ("symbol")
Note: this approach uses the auto-generation of indexes of PostgreSQL (that is, PostgreSQL will identify that that column is unique and add an index to it).
The second way is using CREATE INDEX:
CREATE UNIQUE INDEX "symbol" ON books("symbol");
Last but not least, you can simply omit, in the table creation, the INDEX keyword and let PostgreSQL do the magic (create the index) for you:
CREATE TABLE "books" (
"id" BIGINT NOT NULL,
"creation_date" TIMESTAMP NOT NULL,
"symbol" VARCHAR NOT NULL,
PRIMARY KEY ("id"),
UNIQUE ("symbol")
);
Note: The syntax UNIQUE "symbol"("symbol") could be a confusion made with method 2, as in that method one it is required to specify both the table and the column name (books("symbol")).
Therefore, that is not a PostgreSQL bug, but rather a HeidiSQL one.
Edit: I was able to reproduce the bug and opened an issue on github.
How can I create a case insensitive index or constraint in HSQL running in PostgreSQL (;sql.syntax_pgs=true) mode?
In Postgres, it can be done with lower() or lcase():
CREATE UNIQUE INDEX lower_username_index ON enduser_table ((lcase(name)));
PostgreSQL also has the CITEXT datatype, but unfortunately it does not seem to be supported in HSQL.
I'm currently at HSQL 2.2.8 and PostgreSQL 9.0.5. Alternatively, other in-memory databases that might be a better fit for testing PostgreSQL DDL and SQL?
Thanks in advance!
With HSQLDB, it's better to define a UNIQUE constraint, rather than a unique index.
There are two ways of achieving your aim:
Change the type of the column to VARCHAR_IGNORECASE, then use ALTER TABLE enduser_table ADD CONSTRAINT CONST_1 UNIQUE(name)
Alternatively, create a generated column then create the UNIQUE constraint on this column. `ALTER TABLE enduser_table ADD COLUMN lc_name VARCHAR(1000) GENERATED ALWAYS AS (LCASE(name))'
With both methods, duplicate values in the NAME column are rejected. With the first method, the index is used for searches on the NAME column. With the second method, the index is used for searches on the lc_name column.
(UPDATE) If you want to use the PosgreSQL CITEXT type, define the type in HSQLDB, then use the first alternative.
CREATE TYPE CITEXT AS VARCHAR_IGNORECASE(2000)
CREATE TABLE enduser_table (name CITEXT, ...
ALTER TABLE enduser_table ADD CONSTRAINT CONST_1 UNIQUE(name)