I know how to restore a pg dump into a RDS database if that dump is in my machine, but how could I do it when the dump is available at a remote location, say Amazon S3?
What I want to do is something like this:
pg_restore -h somedomain.us-east-1.rds.amazonaws.com -p 5432 -d databasename -U username https://s3.amazonaws.com/database.dump
But of course this results in
pg_restore: [archiver] could not open input file "https://s3.amazonaws.com/database.dump"
Thanks for your help!
If a filename is not specified, pg_restore will take data from standard input (doc). So, this would work:
wget -O - 'https://s3.amazonaws.com/database.dump' | pg_restore -h somedomain.us-east-1.rds.amazonaws.com -p 5432 -d databasename -U username
Note that you can create an RDS snapshot, then create a new DB instance from that snapshot (doc). In your particular situation, I don't know if that would work better than backing up to S3, but it's worth mentioning.
Related
I have building and testing a psql database on my local machine, and the time is come to move it to a server. I have my dump file, but I'm not sure how to actually get it on the server to restore it there. Obviously something like the following doesn't work:
psql -U user -d datbase < C:/dbbackups/dbbackup.dump
But what is the proper way to get my dump on the server from my local to properly restore it?
You can use pg_dump.
pg_dump -C -h localhost -U localuser dbname | psql -h remotehost -U remoteuser dbname
If you want to use your dump you could of course also transfer your dump file over to the new server securly using e.g. SFTP and then restore it there as you would on your local machine.
I want to take a DB dump from a remote server and then copy this dump to my local.
I tried couple of commands but didn't worked.
Lastly I tried the command below;
pg_dump -h 10.10.10.70 -p 5432 -U postgres -d mydb | gzip > db1.gz
I succesffully take the DB and tried with restore from Pgadmin, it gives;
pg_restore: error: input file appears to be a text format dump. Please use psql
But at this point I can't use psql, I have to use Pgadmin and not sure if I'm able to get successfully DB dump to my local. I mean I can't verify with restore.
How can I take DB dump from remote server to my local?
Thanks!
Use the "custom" format:
pg_dump -F c -h 10.10.10.70 -p 5432 -U postgres -f mydb.dmp mydb
That can be restores with pg_restore and hence with pgAdmin.
You do not have to use pgAdmin. pgAdmin uses pg_restore, and there is nothing that keeps you from using it too.
Using pg_dump under an AWS EC2, I want to make a copy of an existing database but under a new database name, potentially on a different instance.
I tried something like this
pg_dump -C --no-owner --no-tablespaces --dbname=eric --host=fs-......eu-central-1.rds.amazonaws.com --port=5432 --username=sa --password | psql -h fs-...SameOrOtherName.eu-central-1.rds.amazonaws.com --port=5432 --username=sa --password --dbname=new_eric
This comes back with the error
"psql: FATAL: database "new_eric" does not exist"
I am trying to avoid generating a pg_dump file, then modifying it, then reloading it...
I suppose I could "sed" to replace some stuff on the fly...?
I have a database server without much disk space, so I took a backup of the entire db (let's just call it redblue) and saved it locally using the following command (I don't have pg running on my computer):
ssh admin#w.x.y.z "pg_dump -U postgres redblue -h localhost " \
>> db_backup_redblue.sql
I'd like to now restore it to another server (1.2.3.4) which contains an older version of "redblue" database - however wanted to ask if this is right before I try it:
ssh admin#1.2.3.4 "pg_restore -U postgres -C redblue" \
<< db_backup_redblue.sql
I wasn't sure if I need to do -C with the name of the db or not?
Will the above command overwrite/restore the remote database with the file I have locally?
Thanks!
No, that will do nothing good.
You have to start pg_restore on the machine where the dump is. Actually, since this is a plain format dump, you have to use psql rather than pg_restore:
psql -h 1.2.3.4 -U postgres -d redblue -f db_backup_redblue.sql
That requires that there is already an empty database redblue on the target system.
If you want to replace an existing database, you have to use the --clean and --create options with pg_dump.
If you want to use SSL, you'll have to configure the PostgreSQL server to accept SSL connections, see the documentation.
I'd recommend the “custom” format of pg_dump.
Of course, you can do this :) Assuming you use ssh keys to authorize user from source host to destination host.
On the source host you do the pg_dump, then pipe through ssh to destination host like this:
pg_dump -C nextcloud | ssh -i .ssh/pg_nextcloud_key postgres#192.168.0.54 psql -d template1
Hope that helps ;)
I'm trying to pg_dump a SQL database on a remote server in our DMZ. There are 2 problems.
there is not a lot of space left on the remote server so the normal command run to locally backup the database
pg_dump -C database > sqldatabase.sql.bak won't work due to space issues.
I also can't run the other version of pg_dump command to dump database from remote server to local server using:
pg_dump -C -h remotehost -U remoteuser db_name | psql localhost -U localuser db_name
as the server is in our DMZ and port 5432 is blocked. What I'm looking to see is if it is possible to pg_dump the database and immediatly save it (ssh or some other form) as a file to a remote server.
What I was trying was: pg_dump -C testdb | ssh admin#ourserver.com | > /home/admin/testdb.sql.bak
Does anyone know if what i am trying to achieve is possible?
You can connect with ssh to your remote server, do with the connect the pg_dump call and send the output back to stdout of local machine.
ssh user#remote_machine "pg_dump -U dbuser -h localhost -C --column-inserts" \
> backup_file_on_your_local_machine.sql
let's create a backup from remote postgresql database using pg_dump:
pg_dump -h [host address] -Fc -o -U [database user] <database name> > [dump file]
later it could be restored at the same remote server using:
sudo -u postgres pg_restore -C mydb_backup.dump
Ex:
pg_dump -h 67.8.78.10 -Fc -o -U myuser mydb > mydb_backup.dump
complete (all databases and objects)
pg_dumpall -U myuser -h 67.8.78.10 --clean --file=mydb_backup.dump
restore from pg_dumpall --clean:
psql -f mydb_backup.dump postgres #it doesn't matter which db you select here
Copied from: https://codepad.co/snippet/73eKCuLx
You can try to dump part of the table to a file in your local machine like this (assume your local machine has psql installed):
psql -h ${db_host} -p 5432 -U ${db_user} -d ${db_name} \
-c "\copy (SELECT * FROM my_table LIMIT 10000) to 'some_local_file.csv' csv;"
And you can import the exported csv into another db later like this:
COPY my_table FROM '/path/to/some_local_file.csv' WITH (FORMAT csv);
One possible solution - pipe through ssh - has been mentioned.
You also could make your DB server listen on the public inet address, add a hostssl entry for your backup machine to pg_hba.conf, maybe configure a client certificate for security, and then simply run the dump on the client/backup machine with pg_dump -h dbserver.example.com ...
This is simpler for unattended backups.
For the configuration of the connection (sslmode) see also the supported environment variables.
If you would like to periodically backup a database PostgreSQL that is inside of a container in the remote server to your local host by using pg_dump over ssh, this is useful for you:
https://github.com/omidraha/periodic-pgdump-over-ssh