I am working on an ETL where we get data from hive and dump it to Postgres. Just to ensure the data is not corrupt I first store the data in a temp table (created as the main table with all the indexes and constraints) and if the data is validated copy it to the main table.
But it has been taking to long as the data is huge.
Once the data is validated I am now thinking of dropping the main table and then renaming the temp table to the main table.
Will renaming a table in Postgres drop the indexes,constraints and defaults defined on it?
It a word - no, it will not drop any indexes, constraints or defaults. Here's a quick demo:
db=> CREATE TABLE mytab (
id INT PRIMARY KEY,
col_uniq INT UNIQUE,
col_not_null INT NOT NULL DEFAULT 123
);
CREATE TABLE
db=> \d mytab
Table "public.mytab"
Column | Type | Modifiers
--------------+---------+----------------------
id | integer | not null
col_uniq | integer |
col_not_null | integer | not null default 123
Indexes:
"mytab_pkey" PRIMARY KEY, btree (id)
"mytab_col_uniq_key" UNIQUE CONSTRAINT, btree (col_uniq)
db=> ALTER TABLE mytab RENAME TO mytab_renamed;
ALTER TABLE
db=> \d mytab_renamed
Table "public.mytab_renamed"
Column | Type | Modifiers
--------------+---------+----------------------
id | integer | not null
col_uniq | integer |
col_not_null | integer | not null default 123
Indexes:
"mytab_pkey" PRIMARY KEY, btree (id)
"mytab_col_uniq_key" UNIQUE CONSTRAINT, btree (col_uniq)
Related
I want to swap names of two tables with each other. TableA <> TableB. Also atomically to avoid any issues of read/writes. I know I can do that in a transction.
I am creating the table using - CREATE TABLE TableB (LIKE TableA INCLUDING ALL);. Then I am also copying over the FKs from from A to B. I am then doing a INSERT INTO TABLEA.... to copy over the data as well (irrelevant for this). Once all of this is done, I rename the table using ALTER TABLE RENAME to swap the names TableA <> TableB in a transaction as well. So TableA becomes TableA_Old and TableB becomes the new TableA (example below)
However, this doesn't update the references to this new table in other tables, like TableC which still hold a FK against TableA_Old. When I do a \d+ on the newly renamed table TableA (which was TableB) before, I don't see the references. They still point to TableA_Old.
Here is an example
I create TableA
CREATE TABLE TableA (
id serial PRIMARY KEY
);
testdb=> \d TableA
Table "public.tablea"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+------------------------------------
id | integer | | not null | nextval('tablea_id_seq'::regclass)
Indexes:
"tablea_pkey" PRIMARY KEY, btree (id)
I create TableC with reference to TableA
CREATE TABLE TableC(
id serial PRIMARY KEY,
table_a_id INT,
CONSTRAINT table_a_table_c_fk
FOREIGN KEY(table_a_id)
REFERENCES TableA(id)
);
\d TableC
Table "public.tablec"
Column | Type | Collation | Nullable | Default
------------+---------+-----------+----------+------------------------------------
id | integer | | not null | nextval('tablec_id_seq'::regclass)
table_a_id | integer | | |
Indexes:
"tablec_pkey" PRIMARY KEY, btree (id)
Foreign-key constraints:
"table_a_table_c_fk" FOREIGN KEY (table_a_id) REFERENCES tablea(id)
You can see the reference
Now I create TableB that looks like TableA using a function
create or replace function create_table_like(source_table text, new_table text)
returns void language plpgsql
as $$
declare
rec record;
begin
execute format(
'create table %s (like %s including all)',
new_table, source_table);
for rec in
select oid, conname
from pg_constraint
where contype = 'f'
and conrelid = source_table::regclass
loop
execute format(
'alter table %s add constraint %s %s',
new_table,
replace(rec.conname, source_table, new_table),
pg_get_constraintdef(rec.oid));
end loop;
end $$;
select create_table_like('TableA', 'TableB');
\d TableB
testdb=> select create_table_like('TableA', 'TableB');
create_table_like
-------------------
(1 row)
testdb=> \d TableB
Table "public.tableb"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+------------------------------------
id | integer | | not null | nextval('tablea_id_seq'::regclass)
Indexes:
"tableb_pkey" PRIMARY KEY, btree (id)
Now I rename them
BEGIN;
ALTER TABLE TableA RENAME to TableA_OLD;
ALTER TABLE TableB RENAME to TableA;
COMMIT;
Now when I look at the structures, the reference is still to TableA_OLD from TableC
testdb=> \d TableA
Table "public.tablea"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+------------------------------------
id | integer | | not null | nextval('tablea_id_seq'::regclass)
Indexes:
"tableb_pkey" PRIMARY KEY, btree (id)
testdb=> \d TableB
testdb=> \d TableA_old
Table "public.tablea_old"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+------------------------------------
id | integer | | not null | nextval('tablea_id_seq'::regclass)
Indexes:
"tablea_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "tablec" CONSTRAINT "table_a_table_c_fk" FOREIGN KEY (table_a_id) REFERENCES tablea_old(id)
And TableC is pointing at the old table - REFERENCES tablea_old(id)
testdb=> \d TableC
Table "public.tablec"
Column | Type | Collation | Nullable | Default
------------+---------+-----------+----------+------------------------------------
id | integer | | not null | nextval('tablec_id_seq'::regclass)
table_a_id | integer | | |
Indexes:
"tablec_pkey" PRIMARY KEY, btree (id)
Foreign-key constraints:
"table_a_table_c_fk" FOREIGN KEY (table_a_id) REFERENCES tablea_old(id)
Is there a safe way to do this without dropping and recreating the constraints again?
I know i can also update the relfilenode pointer in pg_class and swap them. However, that is risky since TableB could have a slightly different looking schema. So, I am wondering if I can update some other table in the system catalogue or use a DDL that wouldn't require dropping the constraint.
The true identity of an object is its object ID, a number that is used to refer to the object in foreign key constraints an other internal database matters (with the notable exception of most function bodies). Renaming an object does not change its identity. You have to drop and re-create foreign key constraints.
If you use the rename table command then automatically will be renamed all dependencies objects, foreign keys, references linked to this table. For example:
ALTER TABLE public.table1 RENAME TO table2;
I did some research but can't find the exact answer that I look for. Currently I have a primary key column 'id' which is set to serial but I want to change it to bigserial to map to Long in Java layer. What is the best way to achieve this considering this is a existing table? I think my Postgres version is 10.5. Also I am aware that both serial and bigserial are not a data type.
In Postgres 9.6 or earlier the sequence created by a serial column already returns bigint. You can check this using psql:
drop table if exists my_table;
create table my_table(id serial primary key, str text);
\d my_table
Table "public.my_table"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+--------------------------------------
id | integer | | not null | nextval('my_table_id_seq'::regclass)
str | text | | |
Indexes:
"my_table_pkey" PRIMARY KEY, btree (id)
\d my_table_id_seq
Sequence "public.my_table_id_seq"
Type | Start | Minimum | Maximum | Increment | Cycles? | Cache
--------+-------+---------+---------------------+-----------+---------+-------
bigint | 1 | 1 | 9223372036854775807 | 1 | no | 1
Owned by: public.my_table.id
So you should only alter the type of the serial column:
alter table my_table alter id type bigint;
The behavior has changed in Postgres 10:
Also, sequences created for SERIAL columns now generate positive 32-bit wide values, whereas previous versions generated 64-bit wide values. This has no visible effect if the values are only stored in a column.
Hence in Postgres 10+:
alter sequence my_table_id_seq as bigint;
alter table my_table alter id type bigint;
-- backup table first
CREATE TABLE tablenamebackup as select * from tablename ;
--add new column idx
alter table tablename add column idx bigserial not null;
-- copy id to idx
update tablename set idx = id ;
-- drop id column
alter table tablename drop column id ;
-- rename idx to id
alter table tablename rename column idx to id ;
-- Reset Sequence to max + 1
SELECT setval(pg_get_serial_sequence('tablename', 'id'), coalesce(max(id)+1, 1), false) FROM tablename ;
Let's say I have two tables. Is there a way to insert rows into both simultaneously? When I had 1 table I could simply do
INSERT INTO users (first_name, last_name, email) VALUES ('Tom','Prats','tom#tomprats.com'), ('Tim', 'Prats', tim#tomprats.com);
If I now have 1 table that stores email, and an associated table that stores first and last name, how do I insert both at once? Keeping in mind I'm importing a lot of data.
Table "users"
Column | Type | Modifiers
---------------------+---------------+-----------
id | character(36) | not null
email | text |
Indexes:
"pk_users" PRIMARY KEY, btree (id)
Foreign-key constraints:
"fk_users_attributes" FOREIGN KEY (attributes_id) REFERENCES user_attributes(id)
Table "user_attributes"
Column | Type | Modifiers
-----------------+---------------+-----------
id | character(36) | not null
first_name | text |
last_name | text |
Indexes:
"pk_user_attributes" PRIMARY KEY, btree (id)
Referenced by:
TABLE "users" CONSTRAINT "fk_users_attributes" FOREIGN KEY (attributes_id) REFERENCES user_attributes(id)
You can't do this only by insert.
If you must do this in a single line the best way to do it is to write a short function that will make 2 inserts. This will result in a single transaction per call(if this was the main problem).
You can also use Data-Modifying Statements in WITH
I am working in Postgres 9.1 and I want to create a foreign key relationship for two tables that don't currently have one.
These are my tables:
# \d frontend_item;
Table "public.frontend_item"
Column | Type | Modifiers
-------------------+-------------------------+--------------------------------------------------------------------
id | integer | not null default nextval('frontend_prescription_id_seq'::regclass)
presentation_code | character varying(15) | not null
pct_code | character varying(3) | not null
Indexes:
"frontend_item_pkey" PRIMARY KEY, btree (id)
# \d frontend_pct;
Column | Type | Modifiers
------------+--------------------------+-----------
code | character varying(3) | not null
Indexes:
"frontend_pct_pkey" PRIMARY KEY, btree (code)
"frontend_pct_code_1df55e2c36c298b2_like" btree (code varchar_pattern_ops)
This is what I'm trying:
# ALTER TABLE frontend_item ADD CONSTRAINT pct_fk
FOREIGN KEY (pct_code) REFERENCES frontend_pct(code) ON DELETE CASCADE;
But I get this error:
ERROR: insert or update on table "frontend_item" violates
foreign key constraint "pct_fk"
DETAIL: Key (pct_code)=(5HQ) is not present in table "frontend_pct"
I guess this makes sense, because currently the frontend_pct table is empty, while the frontend_item has values in it.
Firstly, is the syntax of my ALTER TABLE correct?
Secondly, is there an automatic way to create the required values in frontend_pct? It would be great if there was some way to say to Postgres "create the foreign key, and insert values into the foreign key table if they don't exist".
Your syntax seems correct.
No, there is not an automatic way to insert the required values.
You can only do it manually before adding the constraint. In your case must be something like
INSERT INTO frontend_pct (code)
SELECT code FROM
(
SELECT DISTINCT pct_code AS code
FROM frontend_item
WHERE pct_code NOT IN (SELECT code FROM frontend_pct)
) AS a;
NOTICE:
The query can be heavy if you have lot of data..
How to see all existed indexes for table? for example given table mytable, how to see his every index with appropriate columns?
Try this SQL
SELECT * FROM pg_indexes WHERE tablename = 'mytable';
In psql use the \d command:
postgres=> create table foo (id integer not null primary key, some_data varchar(20));
CREATE TABLE
postgres=> create index foo_data_idx on foo (some_data);
CREATE INDEX
postgres=> \d+ foo
Table "public.foo"
Column | Type | Modifiers | Storage | Stats target | Description
-----------+-----------------------+-----------+----------+--------------+------------
id | integer | not null | plain | |
some_data | character varying(20) | | extended | |
Indexes:
"foo_pkey" PRIMARY KEY, btree (id)
"foo_data_idx" btree (some_data)
Has OIDs: no
postgres=>
Other SQL tools have other means of displaying this information.