In postgresql update/insert are very slow. So often is faster create a new table with the new field/row and then replace the old table
The thing is I need an autonumeric so in this case I had to create the table first and do the insert later.
Is there a way create an autonumeric field in the select so I can use
CREATE TABLE source.road_nodes AS
SELECT serial_field, node
instead of CREATE TABLE + INSERT.
CREATE TABLE source.road_nodes (
node_id serial,
node TEXT
);
INSERT INTO source.road_nodes (node)
SELECT DISTINCT node
FROM
(
SELECT DISTINCT node_begin AS node
FROM source.vzla_rto
) as node_pool;
In a way this is possible. You can take advantage of the fact that as the PostgreSQL documentation - 8.1.4. Serial Types states:
CREATE TABLE tablename (
colname SERIAL
);
is equivalent to specifying:
CREATE SEQUENCE tablename_colname_seq;
CREATE TABLE tablename (
colname integer NOT NULL DEFAULT nextval('tablename_colname_seq')
);
ALTER SEQUENCE tablename_colname_seq OWNED BY tablename.colname;
Creating a sequence separately and making the next value the default for the column you effectively make it a SERIAL column.
CREATE SEQUENCE road_nodes_node_id_seq;
CREATE TABLE road_nodes AS
SELECT DISTINCT ON (node) nextval('road_nodes_node_id_seq') AS node_id, node FROM vzla_rto;
ALTER TABLE road_nodes ALTER node_id SET NOT NULL, ALTER COLUMN node_id SET DEFAULT nextval('road_nodes_node_id_seq');
You could keep the sequence and just reset it to one when you would recreate the table.
I wouldn't say that this is an elegant way however and probably the easiest way is just to create the table separately.
Related
I hit the int limit on a large table I use.
The table is in single user mode and has no FK constraints.
CREATE TABLE my_table_bigint (LIKE my_table INCLUDING ALL);
ALTER TABLE my_table_bigint ALTER id DROP DEFAULT;
ALTER TABLE my_table_bigint alter column id set data type bigint;
CREATE SEQUENCE my_table_bigint_id_seq;
INSERT INTO my_table_bigint SELECT * FROM my_table;
ALTER TABLE my_table_bigint ALTER id SET DEFAULT nextval('my_table_bigint_id_seq');
ALTER SEQUENCE my_table_bigint_id_seq OWNED BY my_table_bigint.id;
SELECT setval('my_table_bigint_id_seq', (SELECT max(id) FROM my_table_bigint), true);
At this point I tested that I could insert new rows without any problems. Success, I thought.
I went about renaming the tables.
alter table my_table rename my_table_old
alter table my_table_bigint rename my_table
ALTER INDEX post_comments_pkey RENAME TO post_comments_old_pkey
ALTER INDEX post_comments_pkey_bigint RENAME TO post_comments_pkey
Now, when I checked the schema.... the table ID type had changed BACK to integer, instead of bigint.
Copying took about 3 days - so I am really, really hoping that I don't need to do this again. This is postgres10 on RDS.
EDIT
I'm going to take care of this problem like this:
Create a new table - call it my_table_bigint2.
Do this:
CREATE TABLE my_table_bigint2 (LIKE my_table INCLUDING ALL);
ALTER TABLE my_table_bigint2 ALTER id DROP DEFAULT;
ALTER TABLE my_table_bigint2 alter column id set data type bigint;
CREATE SEQUENCE my_table_bigint2_id_seq;
ALTER TABLE my_table_bigint2 ALTER id SET DEFAULT nextval('my_table_bigint2_id_seq');
ALTER SEQUENCE my_table_bigint2_id_seq OWNED BY my_table_bigint2.id;
And start populating that table with the new data. (This is fine given the usecase.)
In the meantime, I'm going to run
ALTER TABLE post_comments alter column id set data type bigint;
And finally, once that's done, I'm going to
INSERT INTO my_table SELECT * FROM my_table_bigint2;
My follow-up question - is this allowed? Will this create some interaction between the sequences? Should I use a new sequence?
I create table in PostgreSQL but I forgot to add auto increment.
How to alter empty Id column in Postgres to add auto increment?
Starting with Postgres 10 it's recommended to use identity columns for this.
You can turn an existing column into an identity column using an ALTER TABLE:
alter table the_table
alter id add generated always as identity;
If you already have data in the table, you will need to sync the sequence:
select setval(pg_get_serial_sequence('the_table', 'id'), (select max(id) from the_table));
You will need to create a sequence owned by that column and set that as the default value.
e.g.
CREATE TABLE mytable (id int);
CREATE SEQUENCE mytable_id_seq OWNED BY mytable.id;
ALTER TABLE mytable ALTER COLUMN id SET DEFAULT nextval('mytable_id_seq');
I have a postgres table creating with the following SQL:
CREATE TABLE mytable (
mytable_id BIGSERIAL NOT NULL,
mytable_char VARCHAR(8) NOT NULL
)
This creates the table as well as an implicit mytable_mytable_id_seq sequence.
Now, after creating 1.000.000 records, I want to split this table into partitioned tables (using inheritance). Because I link refer to the main table from other tables, I want to keep using the IDs from the original table in the new child tables and keep using the sequence.
However, if I do DROP TABLE mytable it also deletes the sequence. How can I keep the sequence when dropping the table?
You need to first remove the association between the column and the sequence:
alter sequence mytable_mytable_id_seq owned by none;
If you now drop the table, the sequence will not be dropped.
Details are in the manual: http://www.postgresql.org/docs/current/static/sql-altersequence.html
An alternative is to create a new sequence and set that to the value of the existing sequence:
create sequence part_seq;
select setval('part_seq', (select nextval('mytable_mytable_id_seq'), false);
I have a table that has one column with unordered value. I want to order this column descending and add a column to record its order. My SQL code is:
select *
into newtable
from oldtable
order by column_name desc;
alter table newtable add column id serial;
Would this implement my goal? I know that rows in PostgreSQL have no fixed order. So I am not sure about this.
Rather than (ab)using a SERIAL via ALTER TABLE, generate it at insert-time.
CREATE TABLE newtable (id serial unique not null, LIKE oldtable INCLUDING ALL);
INSERT INTO newtable
SELECT nextval('newtable_id_seq'), *
FROM oldtable
ORDER BY column_name desc;
This avoids a table rewrite, and unlike your prior approach, is guaranteed to produce the correct ordering.
(If you want it to be the PK, and the prior table had no PK, change unique not null to primary key. If the prior table had a PK you'll need to use a LIKE variant that excludes constraints).
You can first create a new table, sorted based on the column you want to use:
CREATE TABLE newtable AS
SELECT * FROM oldtable
ORDER BY column_name desc;
Afterwards, since you want to order from the largest to the smallest, you can add a new column to your table:
ALTER TABLE newtable ADD COLUMN id serial unique;
Consider the following table with approximately 10M rows
CREATE TABLE user
(
id bigint NOT NULL,
...
CONSTRAINT user_pk PRIMARY KEY (id)
)
WITH (
OIDS=FALSE
)
Then i applied the following alter
ALTER TABLE USER ADD COLUMN BUSINESS_ID VARCHAR2(50);
--OK
UPDATE USER SET BUSINESS_ID = ID; //~1500 sec
--OK
ALTER TABLE USER ALTER COLUMN BUSINESS_ID SET NOT NULL;
ERROR: column "business_id" contains null values
SQL state: 23502
This is very strange since id column (which has been copied to business_id column) can't contain null values since it is the primary key, but to be sure i check it
select count(*) from USER where BUSINESS_ID is null
--0 records
I suspect that this is a bug, just wondering if i am missing something trivial
The only logical explanation would be a concurrent INSERT.
(Using tbl instead of the reserved word user as table name.)
ALTER TABLE tbl ADD COLUMN BUSINESS_ID VARCHAR2(50);
--OK
UPDATE tbl SET BUSINESS_ID = ID; //~1500 sec
--OK
-- concurrent INSERT HERE !!!
ALTER TABLE tbl ALTER COLUMN BUSINESS_ID SET NOT NULL;</code></pre>
To prevent this, use instead:
ALTER TABLE tbl
ADD COLUMN BUSINESS_ID VARCHAR(50) DEFAULT ''; -- or whatever is appropriate
...
You may end up with a default value in some rows. You might want to check.
Or run everything as transaction block:
BEGIN;
-- LOCK tbl; -- not needed
ALTER ...
UPDATE ...
ALTER ...
COMMIT;
You might take an exclusive lock to be sure, but ALTER TABLE .. ADD COLUMN takes an ACCESS EXCLUSIVE lock anyway. (Which is only released at the end of the transaction, like all locks.)
Maybe it wants a default value? Postgresql docs on ALTER:
To add a column, use a command like this:
ALTER TABLE products ADD COLUMN description text;
The new column is initially filled with whatever default value is given (null if you don't specify a DEFAULT clause).
So,
ALTER TABLE USER ALTER COLUMN BUSINESS_ID SET DEFAULT="",
ALTER COLUMN BUSINESS_ID SET NOT NULL;
You cannot do that at the same transaction. Add your column and update it. Then in a separate transaction set the not null constraint.