I'm trying to pg_dump a SQL database on a remote server in our DMZ. There are 2 problems.
there is not a lot of space left on the remote server so the normal command run to locally backup the database
pg_dump -C database > sqldatabase.sql.bak won't work due to space issues.
I also can't run the other version of pg_dump command to dump database from remote server to local server using:
pg_dump -C -h remotehost -U remoteuser db_name | psql localhost -U localuser db_name
as the server is in our DMZ and port 5432 is blocked. What I'm looking to see is if it is possible to pg_dump the database and immediatly save it (ssh or some other form) as a file to a remote server.
What I was trying was: pg_dump -C testdb | ssh admin#ourserver.com | > /home/admin/testdb.sql.bak
Does anyone know if what i am trying to achieve is possible?
You can connect with ssh to your remote server, do with the connect the pg_dump call and send the output back to stdout of local machine.
ssh user#remote_machine "pg_dump -U dbuser -h localhost -C --column-inserts" \
> backup_file_on_your_local_machine.sql
let's create a backup from remote postgresql database using pg_dump:
pg_dump -h [host address] -Fc -o -U [database user] <database name> > [dump file]
later it could be restored at the same remote server using:
sudo -u postgres pg_restore -C mydb_backup.dump
Ex:
pg_dump -h 67.8.78.10 -Fc -o -U myuser mydb > mydb_backup.dump
complete (all databases and objects)
pg_dumpall -U myuser -h 67.8.78.10 --clean --file=mydb_backup.dump
restore from pg_dumpall --clean:
psql -f mydb_backup.dump postgres #it doesn't matter which db you select here
Copied from: https://codepad.co/snippet/73eKCuLx
You can try to dump part of the table to a file in your local machine like this (assume your local machine has psql installed):
psql -h ${db_host} -p 5432 -U ${db_user} -d ${db_name} \
-c "\copy (SELECT * FROM my_table LIMIT 10000) to 'some_local_file.csv' csv;"
And you can import the exported csv into another db later like this:
COPY my_table FROM '/path/to/some_local_file.csv' WITH (FORMAT csv);
One possible solution - pipe through ssh - has been mentioned.
You also could make your DB server listen on the public inet address, add a hostssl entry for your backup machine to pg_hba.conf, maybe configure a client certificate for security, and then simply run the dump on the client/backup machine with pg_dump -h dbserver.example.com ...
This is simpler for unattended backups.
For the configuration of the connection (sslmode) see also the supported environment variables.
If you would like to periodically backup a database PostgreSQL that is inside of a container in the remote server to your local host by using pg_dump over ssh, this is useful for you:
https://github.com/omidraha/periodic-pgdump-over-ssh
Related
I have building and testing a psql database on my local machine, and the time is come to move it to a server. I have my dump file, but I'm not sure how to actually get it on the server to restore it there. Obviously something like the following doesn't work:
psql -U user -d datbase < C:/dbbackups/dbbackup.dump
But what is the proper way to get my dump on the server from my local to properly restore it?
You can use pg_dump.
pg_dump -C -h localhost -U localuser dbname | psql -h remotehost -U remoteuser dbname
If you want to use your dump you could of course also transfer your dump file over to the new server securly using e.g. SFTP and then restore it there as you would on your local machine.
I want to take a DB dump from a remote server and then copy this dump to my local.
I tried couple of commands but didn't worked.
Lastly I tried the command below;
pg_dump -h 10.10.10.70 -p 5432 -U postgres -d mydb | gzip > db1.gz
I succesffully take the DB and tried with restore from Pgadmin, it gives;
pg_restore: error: input file appears to be a text format dump. Please use psql
But at this point I can't use psql, I have to use Pgadmin and not sure if I'm able to get successfully DB dump to my local. I mean I can't verify with restore.
How can I take DB dump from remote server to my local?
Thanks!
Use the "custom" format:
pg_dump -F c -h 10.10.10.70 -p 5432 -U postgres -f mydb.dmp mydb
That can be restores with pg_restore and hence with pgAdmin.
You do not have to use pgAdmin. pgAdmin uses pg_restore, and there is nothing that keeps you from using it too.
i'm tring to backup my database from production machine to a standby machine.
My production machine is a debian (unspecified version) distro
My standby machine is a ubuntu 20.10 distro
After some readings across the internet here is what i was able to accomplish
pg_dump -h 123.123.123.123 -p 5432 my_db > my_db.bak -> check
psql < createDateabase.sql -> check this make a simple drop/create database my_db owner my_db_backup;
psql < my_db.back -> check
so far so good
connect with my sqlworkbench to db -> check
use table imported -> ehm, not check
the problem is the ownership of the tables and sequeces. When imported, postgres is their owner but i want my_db_backup as owner insted.
I have tryed, as a workaround, to
alter table xxx OWNER TO my_db_backup; - > check for each table
alter sequence if exists xxx OWNER TO my_db_backup; -> it takes forever and it is unable to finish;
the command for the final stage (change ownership) is
psql < finalize.sql
and finalize.sql is
\connect my_db;
ALTER TABLE xxx OWNER TO my_db_backup;
ALTER SEQUENCE IF EXISTS seq_xxx OWNER TO my_db_backup; -- this takes forever
what i'm missing?
I would prefer to import directly with the right owner instead of altering owner after, but that's still acceptable as long as i can change all objects in my_db
tnx for the help
I was finally able to accomplish the import with a different user as follow
psql -U my_db_backup < my_db.bak
to avoid asking for password i had to make an entry in pg_hba.conf like this
local my_db my_db_backup trust
and to avoid request for password during the pg_dump i had to add this entry in my production machine
host my_db postgres 456.456.456.456/32 trust
where 456.456.456.456 id the ip of the standby machine.
i also had to change a bit the pg_dump to avoid errors for different user import
pg_dump -h 123.123.123.123 -p 5432 --no-owner --no-acl my_db > my_db.bak
so the complete script looks like
pg_dump -h 123.123.123.123 -p 5432 --no-owner --no-acl my_db > my_db.bak
psql < createDatabase.sql
psql -U my_db_backup < my_db.bak
EDIT: to make the script usable from any user there are a few things in need to be done
pg_dump -h 123.123.123.123 -p 5432 -U postgres --no-owner --no-acl -w my_db > my_db.bak
psql -U postgres < createDatabase.sql
psql -U my_db_backup < my_db.bak
Adding -U postgres is needed only because pg_dump try to login with the current user so if you are running the script as, for example, root pg_dump and psql will try to login as root and will obviously fail.
The other thing is the -w option that is needed to make pg_dump not ask for password.
Is it possible to use pg_dump to backup to another server on the LAN?
pg_dump -U username dbname > rich#workstation.local/users/rich/pg_backups/bkup-dbname.sql
So from a server to rich's workstation pg_backups directory? Is this doable or does rich need to pull the backup from his workstation?
There are many ways to do this and here are a couple. Assuming you have ssh access to the remote host:
pg_dump -Fc myDb |ssh remoteHost 'cat >/tmp/dumpOfMyDb'
If you have the remote host mounted as an NFS share on your localhost, then you can just run:
pg_dump -d myDb -Fc -f /path/to/nfs/mount
Tip: -Fc has compression enabled.
I have a database server without much disk space, so I took a backup of the entire db (let's just call it redblue) and saved it locally using the following command (I don't have pg running on my computer):
ssh admin#w.x.y.z "pg_dump -U postgres redblue -h localhost " \
>> db_backup_redblue.sql
I'd like to now restore it to another server (1.2.3.4) which contains an older version of "redblue" database - however wanted to ask if this is right before I try it:
ssh admin#1.2.3.4 "pg_restore -U postgres -C redblue" \
<< db_backup_redblue.sql
I wasn't sure if I need to do -C with the name of the db or not?
Will the above command overwrite/restore the remote database with the file I have locally?
Thanks!
No, that will do nothing good.
You have to start pg_restore on the machine where the dump is. Actually, since this is a plain format dump, you have to use psql rather than pg_restore:
psql -h 1.2.3.4 -U postgres -d redblue -f db_backup_redblue.sql
That requires that there is already an empty database redblue on the target system.
If you want to replace an existing database, you have to use the --clean and --create options with pg_dump.
If you want to use SSL, you'll have to configure the PostgreSQL server to accept SSL connections, see the documentation.
I'd recommend the “custom” format of pg_dump.
Of course, you can do this :) Assuming you use ssh keys to authorize user from source host to destination host.
On the source host you do the pg_dump, then pipe through ssh to destination host like this:
pg_dump -C nextcloud | ssh -i .ssh/pg_nextcloud_key postgres#192.168.0.54 psql -d template1
Hope that helps ;)