I have made a backup of my specific tables that I want to restore into a new database using:
call pg_dump -Fc -h server -d database -U user -p password -v -f dump.sql -t public.table1 -t public.table2
And I have no problems.
I then want to restore the dump by creating a new database with pg_restore using:
call pg_restore --clean --create -d temp -h server -p password -U user dump.sql
This gives me a "database temp does not exist" error. This should have created the database and restored it as far as I understand.
I However then create the "temp" database manually and run:
call pg_restore --clean --create -d temp -h server -p password -U user dump.sql
This follows through, but does not create the tables and gives me an error "relation table1" and "relation table2" does not exist and only creates the corresponding id_sequences for the two tables.
What I actually want is to not have to manually create the new database and that all tables in the backup is restored into a brand new database via pg_restore using:
call pg_restore --clean --create -d temp -h server -p password -U user dump.sql
as I understand it.
In order to create the database temp, pg_restore needs to be connected to a different database first.
So it won't do to use -d temp; you must specify an existing database, typically postgres.
pg_restore will connect to that database, issue CREATE DATABASE temp, connect to temp and proceed to restore the data.
Related
I have take backup by pg_dumpall > test.out
and test.out successfully generated, hence backup completed.
I have used command psql -f test.out postgres for restore
But got following errors with restoring backup:
databases already exists
relation "products" already exists
duplicate key value violates unique constraint "products_pkey"
I actually want to replace the data in the existing db with backup. How to do that?
The problem is that the database you're trying to restore already exists.
You can run a DROP DATABASE database_name command that will delete your existing database and then you can run your test.out file.
Or you can run pgdumpall --clean > test.out and then run the resulting file. The clean flag will make the resulting files have the DROP DATABASE command in them.
Do you use the bellow command ?
psql -h localhost -U [login role] database_name -f /home/database.backup
I think a flow like this might help, because we don't want drop the database each time we call the backup file.
First, we need to create a backup file using the --format=custom [-Fc] to restore it using pg_restore. We can use a connection string postgresql://<user>:<pass>#localhost:5432/<dbname> and replace <user>, <pass>, and <dbname> with your information.
pg_dump -v -Fc \
postgresql://<user>:<pass>#localhost:5432/<dbname> \
> db-20211122-163508.sql
To restore we will call it using --clean [-c] and --create [-C] to drop the database before restoring. Replace <user>, <host>, <port>, and <dbname> with your information.
pg_restore -vcC \
-U <user> \
-h <host> \
-p <port> \
-d <dbname> \
< db-20211122-163508.sql
This way you don't need to use clean when you create the backup file.
I have a users-production database on 157.157.35.333
I would like to clone it to another host as users-sandbox
here is what I tried:
PRODUCTION_HOST=111.111.11.111
SANDBOX_HOST=222.222.22.222
echo "creating db on sanbox"
psql -h ${SANDBOX_HOST} -U postgres -c "CREATE DATABASE \"users-sandbox\";"
pg_dump -h ${PRODUCTION_HOST} -U postgres -d users-production -F c -b -v | \
pg_restore -C -c -h ${SANDBOX_HOST} -U postgres -d users-sandbox -v
but this creates the database with the old name
how do I create with a new name?
https://www.postgresql.org/docs/current/static/app-pgrestore.html
-d dbname
Connect to database dbname and restore directly into the database.
but!
-C
--create
Create the database before restoring into it. If --clean is also
specified, drop and recreate the target database before connecting to
it.
When this option is used, the database named with -d is used only to
issue the initial DROP DATABASE and CREATE DATABASE commands. All data
is restored into the database name that appears in the archive.
so remove -C from pg_restore arguments...
(formatting of quotes mine)
I ONLY dump my databases as *.sql files not *.dump files. As a result NONE of the pg_restore commands work. I've been reading through answers and I swear most people have a reading disability lol
I am asking for the equivalent in psql for a common pg_restore commandLine method to restore a database.
I have no intention of dumping my databases as *.dump.
my question is this:
what is the equivalent to:
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U myuser -d my_db db/latest.dump
using psql
so...
something along the lines of:
psql --verbose --clean --no-acl --no-owner -h localhost -U myuser -d my_db db/latest.sql
With a SQL dump you need to decide whether you want to drop target objects, when dumping the database, not when importing it.
So, you need to use:
pg_dump --clean ....
Then the SQL dump will contain the necessary DROP statements.
Another option is to run drop owned by current_user before doing the import. This however requires that everything is owned by the user doing the import (so you can't run the import as e.g. postgres)
This can be combined with running the SQL dump:
psql -U your_user -d your_db -c 'drop owned by current_user' -f your_dump.sql
I use the postgres today
and got a problem
I dump the database that way
pg_dump zeus_development -U test > zeus_development.dump.out
what if I wnat to restore to another database zeus_production
How could I do?
Simple, first create your database using template0 as your template database:
createdb -U test -T template0 zeus_production
Then, restore your dump on this database:
psql -U test zeus_production -f /path/to/zeus_development.dump.out
When restoring, always use template0 explicit, as it is always an empty and unmodifiable database. If you don't use an explicit template, PostgreSQL will assume template1, and if it has some objects, like a table or function that your dumped database already has, you will get some errors while restoring.
Nonetheless, even if you were restoring on a database with the same name (zeus_development) you should create (or recreate) it the same way. Unless you used -C option while dumping (or -C of pg_restore if using a binary dump), which I don't recommend, because will give you less flexibility (like restoring on a different database name).
The PostgresSQL documentation has influenced me to use the custom format. I've been using it for years and it seems to have various advantages but your mileage may vary. That said, here is what worked for me:
pg_restore --no-owner --dbname postgres --create ~/Desktop/pg_dump
psql --dbname postgres -c 'ALTER DATABASE foodog_production RENAME TO foodog_development'
There was no foodog_development nor foodog_production databases existing before the sequence.
This restores the database from the dump (~/Desktop/pg_dump) which will create it with the name it was dumped as. The rename names the DB to whatever you want.
The --no-owner may not be needed if your user name is the same on both machines. In my case, the dump was done as user1 and the restore done as user2. The new objects need to be owned by user2 and --no-owner achieves this.
Isn't it easier to simply do the following?
createdb -U test -T zeus_development zeus_production
This has an answer on dba.stackexchange, which I reproduce here:
Let's define a few variables to make the rest easier to copy/paste
old_db=my_old_database
new_db=new_database_name
db_dump_file=backups/my_old_database.dump
user=postgres
The following assumes that your backup was created with the "custom" format like this:
pg_dump -U $user -F custom $old_db > "$db_dump_file"
To restore $db_dump_file to a new database name $new_db :
dropdb -U $user --if-exists $new_db
createdb -U $user -T template0 $new_db
pg_restore -U $user -d $new_db "$db_dump_file"
Here's a hacky way of doing it, that only works if you can afford the space and time to use regular .sql format, and if you can safely sed out your database name and user.
$ pg_dump -U my_production_user -h localhost my_production > my_prod_dump.sql
$ sed -i 's/my_production_user/my_staging_user/g' my_prod_dump.sql
$ sed -i 's/my_production/my_staging/g' my_prod_dump.sql
$ mv my_prod_dump.sql my_staging_dump.sql
$ sudo su postgres -c psql
psql> drop database my_staging;
psql> create database my_staging owner my_staging_user;
psql> \c my_staging;
psql> \i my_staging_dump.sql
If your dump does not include the name, the restore will use the DB defined in DESTINATION. Both SOURCE and DESTINATION are Connection URLs.
Dump without --create
pg_dump \
--clean --if-exists \
--file ${dump_path} \
--format=directory \
--jobs 5 \
--no-acl \
--no-owner \
${SOURCE}
Restore without --create
pg_restore \
--clean --if-exists \
--dbname=${DESTINATION} \
--format=directory \
--jobs=5 \
--no-acl \
--no-owner \
$dump_path
I'm able to push my database from my local machine to my heroku app with pg:transfer like this:
heroku pg:transfer --from postgres://localhost/idx_map_development --to aqua
But I don't want to upload the whole database, I only want to upload the properties table, is this possible using pg:transfer? If so how would the above line look?
did it with pg_dump:
pg_dump -Fc --no-acl --no-owner -h localhost -U username -t properties idx_map_development > properties.dump
The file called properties.dump gets only the data for the properties table and I upload the file to an Amazon S3 bucket and can then push it to heroku with this:
heroku pgbackups:restore DATABASE 'https://s3.amazonaws.com/bucket_name/properties.dump'
The other tables on my heroku db are unaffected. You don't need to specify the properties table in the pgbackups:restore command.
As was mentioned by #eugjill, this doesn't work anymore. So first you do the dump, as described by #railsy:
pg_dump -Fc --no-acl --no-owner -h localhost -U username -t properties idx_map_development > properties.dump
Then instead of using pgbackups, use the pg_restore:
PGPASSWORD=<PASSWORD> pg_restore --verbose --no-acl --no-owner -h <HOST> -U <USER> -d <DATABASE> -p <PORT> properties.dump
The pg_dump above worked for me as well; though I needed all of the options described.
I had two tables; a user table and a transactions table.
pg_dump -t transactions db/development > transactions_clobber.bak
When I restored from this; the user table was lost.
pg_dump -Fc --no-acl --no-owner -t transactions db/development > transactions_works.bak
works as advertised.