What I want is a way to compare two versions of the same database and generate a sql file that updates DatabaseOld to DatabaseNew. Mysqldbcompare sounds like it might do that, but it is not clear to me how.
Has anyone used mysqldbcompare for a usecase like this?
I have seen some attempts to diff dumps (eg Compare two MySQL databases) and such, but I'm looking for something a little more robust and reliable than that.
I see you never got a true answer so maybe this will help someone that is searching. mysqldbcompare will at least come close to what you are after. The issue I note so far is it will let you know tables are missing but not provide the CREATE statements. Another potential issue may be altering AUTO_INCREMENT fields. Here is the statement I am using to get just the schema differences (I don't care about data).
mysqldbcompare --server1=root:password#localhost test:prod --changes-for=server2 --difftype=sql --show-reverse --run-all-tests --skip-row-count --skip-data-check > C:\Temp\DBCompare.sql
This will provide output like this:
# WARNING: Using a password on the command line interface can be insecure.
# server1 on localhost: ... connected.
# Checking databases test and prod on server1
#
# Transformation for --changes-for=server2:
#
ALTER DATABASE prod CHARACTER SET latin1 COLLATE = latin1_swedish_ci;
#
# Transformation for reverse changes (--changes-for=server1):
#
# ALTER DATABASE test CHARACTER SET utf8 COLLATE = utf8_general_ci;
#
# WARNING: Objects in server1.test but not in server1.prod:
# PROCEDURE: my_test_noout
# TABLE: test
# PROCEDURE: my_test
# TABLE: customer2
# TABLE: ordgdnames
#
# Defn Row Data
# Type Object Name Diff Count Check
# -------------------------------------------------------------------------
# TABLE arahst FAIL SKIP SKIP
#
# Transformation for --changes-for=server2:
#
ALTER TABLE `prod`.`arahst`
ADD INDEX idx_arahst_CustId (CustId);
#
# Transformation for reverse changes (--changes-for=server1):
#
# ALTER TABLE `test`.`arahst`
# DROP INDEX idx_arahst_CustId;
#
# TABLE aropen pass SKIP SKIP
# TABLE brkitem FAIL SKIP SKIP
#
# Transformation for --changes-for=server2:
#
ALTER TABLE `prod`.`brkitem`
DROP INDEX ItemId,
DROP PRIMARY KEY,
ADD UNIQUE INDEX idx_brkitem_ItemId (ItemId);
#
# Transformation for reverse changes (--changes-for=server1):
#
# ALTER TABLE `test`.`brkitem`
# DROP INDEX idx_brkitem_ItemId,
# ADD PRIMARY KEY(`ItemId`),
# ADD UNIQUE INDEX ItemId (ItemId);
#
# TABLE category FAIL SKIP SKIP
#
# Transformation for --changes-for=server2:
#
ALTER TABLE `prod`.`category`
DROP INDEX GrpId,
DROP INDEX CatId,
ADD INDEX idx_category_GrpId (GrpId),
ADD UNIQUE INDEX idx_category_CatId (CatId);
#
# Transformation for reverse changes (--changes-for=server1):
#
# ALTER TABLE `test`.`category`
# DROP INDEX idx_category_GrpId,
# DROP INDEX idx_category_CatId,
# ADD INDEX GrpId (GrpId),
# ADD INDEX CatId (CatId);
#
# TABLE cusord pass SKIP SKIP
# TABLE customer FAIL SKIP SKIP
#
# Transformation for --changes-for=server2:
#
ALTER TABLE `prod`.`customer`
DROP INDEX ID,
DROP INDEX CustId,
ADD PRIMARY KEY(`ID`),
AUTO_INCREMENT=2200037;
#
# Transformation for reverse changes (--changes-for=server1):
#
# ALTER TABLE `test`.`customer`
# DROP PRIMARY KEY,
# ADD INDEX ID (ID),
# ADD UNIQUE INDEX CustId (CustId),
# AUTO_INCREMENT=11048819;
#
Related
In database dump created with pg_dump, some tables have DEFAULTs in the CREATE TABLE statement, i.e.:
CREATE TABLE test (
f1 integer DEFAULT nextval('test_f1_seq'::regclass) NOT NULL
);
But others have an additional ALTER statement:
ALTER TABLE ONLY test2 ALTER COLUMN f1 SET DEFAULT nextval('test2_f1_seq'::regclass);
What is the reason of this? All sequential fields were created with type SERIAL, but in the dump they look different, and I can't guess any rule for this.
The difference must be that in the first case, the sequence is “owned” by the table column.
You can specify this dependency using the OWNED BY clause when you create a sequence. A sequence that is owned by a column will automatically be dropped when the column is.
If a sequence is implicitly created by using serial, it will be owned by the column.
I am new in PostgreSQL and I am working with this database.
I got a file which I imported, and I am trying to get rows with a certain ID. But the ID is not defined, as you can see it in this picture:
so how do I access this ID? I want to use an SQL command like this:
SELECT * from table_name WHERE ID = 1;
If any order of rows is ok for you, just add a row number according to the current arbitrary sort order:
CREATE SEQUENCE tbl_tbl_id_seq;
ALTER TABLE tbl ADD COLUMN tbl_id integer DEFAULT nextval('tbl_tbl_id_seq');
The new default value is filled in automatically in the process. You might want to run VACUUM FULL ANALYZE tbl to remove bloat and update statistics for the query planner afterwards. And possibly make the column your new PRIMARY KEY ...
To make it a fully fledged serial column:
ALTER SEQUENCE tbl_tbl_id_seq OWNED BY tbl.tbl_id;
See:
Creating a PostgreSQL sequence to a field (which is not the ID of the record)
What you see are just row numbers that pgAdmin displays, they are not really stored in the database.
If you want an artificial numeric primary key for the table, you'll have to create it explicitly.
For example:
CREATE TABLE mydata (
id integer GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
obec text NOT NULL,
datum timestamp with time zone NOT NULL,
...
);
Then to copy the data from a CSV file, you would run
COPY mydata (obec, datum, ...) FROM '/path/to/csvfile' (FORMAT 'csv');
Then the id column is automatically filled.
I have a table with existing data. Is there a way to add a primary key without deleting and re-creating the table?
(Updated - Thanks to the people who commented)
Modern Versions of PostgreSQL
Suppose you have a table named test1, to which you want to add an auto-incrementing, primary-key id (surrogate) column. The following command should be sufficient in recent versions of PostgreSQL:
ALTER TABLE test1 ADD COLUMN id SERIAL PRIMARY KEY;
Older Versions of PostgreSQL
In old versions of PostgreSQL (prior to 8.x?) you had to do all the dirty work. The following sequence of commands should do the trick:
ALTER TABLE test1 ADD COLUMN id INTEGER;
CREATE SEQUENCE test_id_seq OWNED BY test1.id;
ALTER TABLE test1 ALTER COLUMN id SET DEFAULT nextval('test_id_seq');
UPDATE test1 SET id = nextval('test_id_seq');
Again, in recent versions of Postgres this is roughly equivalent to the single command above.
ALTER TABLE test1 ADD COLUMN id SERIAL PRIMARY KEY;
This is all you need to:
Add the id column
Populate it with a sequence from 1 to count(*).
Set it as primary key / not null.
Credit is given to #resnyanskiy who gave this answer in a comment.
To use an identity column in v10,
ALTER TABLE test
ADD COLUMN id { int | bigint | smallint}
GENERATED { BY DEFAULT | ALWAYS } AS IDENTITY PRIMARY KEY;
For an explanation of identity columns, see https://blog.2ndquadrant.com/postgresql-10-identity-columns/.
For the difference between GENERATED BY DEFAULT and GENERATED ALWAYS, see https://www.cybertec-postgresql.com/en/sequences-gains-and-pitfalls/.
For altering the sequence, see https://popsql.io/learn-sql/postgresql/how-to-alter-sequence-in-postgresql/.
I landed here because I was looking for something like that too. In my case, I was copying the data from a set of staging tables with many columns into one table while also assigning row ids to the target table. Here is a variant of the above approaches that I used.
I added the serial column at the end of my target table. That way I don't have to have a placeholder for it in the Insert statement. Then a simple select * into the target table auto populated this column. Here are the two SQL statements that I used on PostgreSQL 9.6.4.
ALTER TABLE target ADD COLUMN some_column SERIAL;
INSERT INTO target SELECT * from source;
ALTER TABLE test1 ADD id int8 NOT NULL GENERATED ALWAYS AS IDENTITY;
How to guarantee a uniqueness of any integer from the two columns / array?
Example: I create a table and insert one row in it:
CREATE TABLE mytest(a integer NOT NULL, b integer NOT NULL);
INSERT INTO mytest values (1,2);
What UNIQUE INDEX should I create to not allow add any of the following values
INSERT INTO mytest values (1,3); # because 1 is already there
INSERT INTO mytest values (3,1); # because 1 is already there
INSERT INTO mytest values (2,3); # because 2 is already there
INSERT INTO mytest values (3,2); # because 2 is already there
I can have array of two elements instead of two columns if it helps somehow.
Surely, I can invent some workaround, the following come into my mind:
create separate table for all numbers, have unique index there, and add values to two tables with transaction. If the number is not unique, it won't be added to the second table, and transaction fails
add two rows instead of one, and have additional field for id-of-the-pair.
But I want to have one table and I need one row with two elements in it. Is that possible?
You can use exclusion constraint on table along with intarray to quickly perform search of overlapping arrays:
CREATE EXTENSION intarray;
CREATE TABLE test (
a int[],
EXCLUDE USING gist (a gist__int_ops WITH &&)
);
INSERT INTO test values('{1,2}');
INSERT INTO test values('{2,3}');
>> ERROR: conflicting key value violates exclusion constraint "test_a_excl"
>> DETAIL: Key (a)=({2,3}) conflicts with existing key (a)=({1,2}).
I have a table linelevelpmts with a column seq (Int4) which is to be used as a sequence.
I know I can delete the column and recreate it as type serial, but can I modify the existing column to be used as a sequence.
ALTER TABLE "public"."linelevelpmts" ALTER COLUMN "seq" SET DEFAULT nextval('linelevelpmts_seq_seq'::regclass);
This code generates an error: Relation linelevelpmts_seq_seq does not exist.
This code generates an error: Relation linelevelpmts_seq_seq does not exist.
Well you need to first create the sequence you want to use for the default value:
create sequence linelevelpmts_seq_seq;
ALTER TABLE public.linelevelpmts
ALTER COLUMN seq SET DEFAULT nextval('linelevelpmts_seq_seq'::regclass);
If you want the same effect as if it was created as serial you also need to change the "owner" of the sequence:
alter sequence linelevelpmts_seq_seq owned by linelevelpmts.seq;
Edit
Igor's comment is a good one: if you already have values in the column seq you should adjust the starting value of the sequence:
select setval('linelevelpmts_seq_seq', (select max(seq) from linelevelpmts));