I'm having some issue with my migrations in Heroku. I've tried several things but none seem to work.
I feel like I should just drop the table that's causing the issue, which will delete all the data in production.
If I drop the table, and re-created the table, will I be able to restore all of the data I lost? Because I will backup my database on Heroku before I drop the table.
Thanks!
You should run a backup with
pg_dump -h hostname -p 5432 -U username -F c -t mytable -f dumpfile mydatabase
Then, after you have dropped and re-created the table, you can restore the data with
pg_restore -h hostname -p 5432 -U username -a -d mydatabase dumpfile
However, this will not work if the table structure has changed.
In that case, you might want to use COPY directly to write the data to a file and restore them from there.
Let's for example assume you plan to add another column. Then you could dump with
COPY (SELECT *, NULL FROM mytable) TO '/file/on/dbserver';
After the table was created with the new column, you can
COPY mytable FROM '/file/on/dbserver';
The new column will be filled with the NULL values.
Modify this basic recipe for more fancy requirements.
Related
After making some schema change in table message in postgresql 13 database, the table is backed up in pgadmin4 in a file named message.sql. The schema change needs to be populated in another database. What I did is to drop the table message in that database. But I have hard time to re-create table message from the backup file messages.sql (not using CREATE table message ....). I use pg_restore to restore data but was not successful for re-creating table schema. Here is command I tried with no luck:
pg_restore --data-only -h localhost -U postgres -W -d dynamo -t messages /home/download/messages.sql
pg_restore --clean -h localhost -U postgres -W -d dynamo -t messages /home/download/messages.sql
I also tried to create an blank table (no column) called messages in the database and repeated above command, again without luck.
Here is how a new table is added to the current database:
backup the current db on the server with pg_dump
create the new database on dev PC with addition of new table
On dev PC, use pgadmin to restore the backup file in step 1 to new database created in step 2.
backup in pgadmin and create a .backup file for db in step 3.
on server, use pg_restore to overwrite the current db with new database backup file created in step 4:
pg_restore --clean -U postgres --dbname=mydb -W -h localhost --verbose /home/download/mydb.backup
open psql terminal to verify the new table
We are migrating some products, one of the steps is to migrate the product databases.
I have steps to
export the existing DB pg_dump --no-owner --clean --blobs --no-privileges -U dbuser old_dbname -f bkpfile.sql
import the dump to a different DB psql -U dbuser2 new_dbname -f bkpfile.sql
The problem is the old database contains statement COMMENT ON DATABASE old_dbname IS 'Rxxxxx';
The new DB user must not have permissions on the old database and imho it's not good to refer the old database name anyway in the dump.
Is there a way to create a complete DB dump without the COMMENT ON DATABASE statement?
Edit:
PostgreSQL 9.6
Steps to reproduce:
CREATE DATABASE testdb;
COMMENT ON DATABASE testdb IS 'some comment';
CREATE TABLE xx (id int);
and then dump the database, the dump contains reference to the database name COMMENT ON DATABASE testdb IS 'some comment'; which prevents importing the backup to a new database
pg_dump --no-owner --clean --blobs --no-privileges testdb
We could manually remove the comment statement or filter the comment using different tools (grep), but manual intervention or text-based filtering on top of the backup could cause data corruption.
This comment is only dumped in PostgreSQL versions below v11. See this entry in the release notes:
pg_dump and pg_restore, without --create, no longer dump/restore database-level comments and security labels; those are now treated as properties of the database.
9.6 will go out of support soon anyway, so this is a good opportunity to upgrade.
I have two servers (and two databases each one from a server on pgAdmin):
The first server is for testing and the second one is the real server that the client will work on it.
The two databases have same tables: 9 tables on the first one and the same 9 tables on the second one.
But on the test server, I have constraint keys (Check, foreign, unique, not null) that I have tested and ready to be deployed on the server 2.
How i can copy only those constraints?
Thank you so much.
The simplest way to do this is to use a command-line pg_dump tool, which is included in all PostgreSQL installations.
Create a backup of the target database:
$ pg_dump -h /var/run/postgresql/ -d prod -Fd -f prod.pg_dump
Create a backup of the source database schema:
$ pg_dump -h /var/run/postgresql/ -d test --section=pre-data --section=post_data -Fd -f test_schema.pg_dump
Drop the target database:
postgres=# drop database prod;
Recreate empty target database:
postgres=# create database prod;
Import to the target database:
test table definitions:
$ pg_restore -d prod --section=pre-data test_schema.pg_dump
prod data:
$ pg_restore -d prod --section=data prod.pg_dump
test constraints and indexes:
$ pg_restore -d prod --section=post-data test_schema.pg_dump
Analyze the target database:
prod=> analyze;
This can be optimized with parallel restore with proper usage of pg_restore -j.
You might consider a proper schema change management solution for the future.
I am taking the dump of postgres database using "pg_dump database_name > backup.sql". Later on I am doing some modifications in the original database(database_name) and then I am restoring the data from the backup file(backup.sql). But the result is that, the database doesn't gets restored to the original state, instead it adds the original data to the modified data(modified + original).I just want it to restore to the original state, shall i delete all the data from the database before restoring it from the backup file, since it gives the original state of the database. Or is there any other way to do this?
The default format fo pg_dump is plain, so it creates a COPY statement. Hence when you psql backup.sql you just run those copy over existing data. To rewrite data, you should either drop tables first or pg_dump -F c and pg_restore -c.
Warning - in both cases it will destroy old data (this seems what you want though)
-c
--clean Clean (drop) database objects before recreating them. (Unless --if-exists is used, this might generate some harmless error messages, if any objects were not present in the destination database.)
As #Craig Ringer suggests, drop/recreate from backup would be much easier and cleaner. To drop database you run DROP DATABASE au - note that there should be no connected users to success. Then you have to create db: CREATE DATABASE au and run psql -f backup.sql -d au
Take the dump with -c option: pg_dump -c database_name > backup.sql. See pg_dump docs.
I would like to copy two tables from database A to database B, in postgres
how can I do it using pg_dump without losing the previous tables and data in database B ?
I read some answers in Stack Overflow suggesting using pg_dump but in the documentation page I read?
The idea behind this dump method is to generate a text file with SQL
commands that, when fed back to the server, will recreate the database
in the same state as it was at the time of the dump
Doesn't that mean it will delete the previous data in database B?
If someone could tell me step by step solution to move two tables in database A to database B without losing any previous data in Database B, it would be helpful.
I found the answer to my question :
sudo -u OWNER_USER pg_dump -t users databasename1 | sudo -u OWNER_USER psql databasename2
if you pg_restore a database into b database, of course a will replace b. instead pick specific table you would like to restore using pg_restore -t
you could pg_restore to different schema, by using -O (no_owner)
so let say
pg_dump -Fc -f dump.dmp -v -h host -U user_login -n schema_to_dump
you can
pg_restore -v -h host -U user_login -n schema_to_import -a --disable-triggers dump.dmp