I'm using pg_dump in script which runs everyday in night to take backup of multiple DBs. we have around 20 DBs and we noticed that one DB backup takes more than 6 hrs and then got failed and after that other DBs backup started. is there anyway to fine tune this? instead to pg_dump wait for many hrs to finish that broken DB backup?
Thanks
Run the dump scripts for each database in separate scripts so they are not dependent on each other. Otherwise use the third party solutions suggested in comment.
Related
This question already has answers here:
Export and import table dump (.sql) using pgAdmin
(6 answers)
Closed 1 year ago.
Let I first state that I am not a DBA-guy but I do have a question regarding restoring remote databases using PG Admin.
I have this PG Admin tool (v4.27) running in a Docker container and I use this portal to maintain two separate Postgress databases, both running in a Docker container as well. I installed PG Agent in both database containers and run scheduled daily backup's, defined via PG Admin and stored in the container of each corresponding databases. So far so good.
Now I want to restore one of these databases by using the latest daily backup file (*.sql), but the Restore Dialog of PG Admin only looks for files stored locally (the PG Admin container)?
Whatever I tried or searched for on the internet, to me it seems not possible to show a list of remote backup files in PG Admin or run manually a remote SQL file. Is this even possible in PG Admin? Running psql in the query editor is not possible (duh ...) and due to not finding the remote SQL-restore file I have no clue how to run this code within PG Admin on the remote corresponding database container.
The one and only solution so far I can think of, is scheduling a restore which has no calendar and should be triggered manually when needed, but it's not the prettiest solution.
Do I miss something or did I overlook the right documentation or have I created a silly, unmaintainable solution?
Thanks in advance for thinking along and kind regards,
Aad Dijksman
You cannot restore a plain format dump (an SQL script) with pgAdmin. You will have to use psql, the command line client.
COPY statements and data are mixed in such a dump, and that would make pgAdmin choke.
The solution by #Laurenz Albe points out that it is best to use the command line psql here, and that would be my first go-to.
However, if for whatever reason you don't have access to the command line and are only able to connect to this database via pgadmin, there is another solution which you can find here:
Export and import table dump (.sql) using pgAdmin
I recommend looking at the solution by Tomas Greif.
I'm giving a try on PoWA, but I've got a little problem.
My Postgres database is running on AWS RDS.
PoWA needs HypoPG in order to suggest indexes.
But RDS doesn't support HypoPG extension. So I had to install PoWA at my backup database (outside RDS).
The problem is: PoWA isn't analyzing the restored database. It can't recognizes any data. If I execute some SQL queries manually it works though.
Is there something that I can be missing?
And, when I tried Ankane Dexter, I could show it the log files path (dumped at the backup database in parallel). Is there a way to do so in PoWA?
Thanks.
I know the function "pg_dump" for backup and restore.
But I never tried because so scared.
Question is simple. Can I use that function for graph data?
Or Is the other function supported for that? There's no information in their documentation.
You may refer the postgreSQL pg_dump document coz nothing different from doing bakcup on postgreSQL.
I referred the guide to create dump script with crontab and both dump and restore worked fine.
In my case, I used pg_dump for creating dump file and restore it with psql. You may choose pg_restore instead if necessary.
agens#karl ~] pg_dump --port=5432 --username=agens --file=agens.dump agens
agens#karl ~] psql --port=5432 --username=agens --dbname=agens2 -f agens.dump
However, I no longer use pg_dump for backup task due to the incremental bakcup requirement. So I googled available backup OSS for postgresql. Among the options I searched, pg_rman is currently what I am using.
It made me easier to build a scheduling script for archive backup every 6 hours, incremental backup every day and full backup every week and those jobs are working properly more than 2 months so far.
Restoring the data on other severs is tested successfully as well.
Hope this helpful for you.
A linux VM with postgres 9.4 was hacked into. (Two processes taking 100% cpu, weird files in /tmp, did not reoccur after kill(s) and restart.) It was decided to install the system from scratch on a new machine (with postgres 9.6). The only data needed was in one of postgres databases. A pg_dump of the database was made after the attack.
Regardless of whether the data - the tables/rows/etc. - were modified during the attack: is it safe to restore the database in the new system?
I consider using pg_restore with the -O option (ignores the user permissions)
The two dangers are:
important data could have been modified
back doors could have been installed in your database
With the first, you're on your own how to verify that your data are ok. The safest thing would be to use a backup from before the machine was compromized, but this would mean data loss.
For the second, I would run a pg_dumpall -s and spend a day reading it carefully. Compare it with a dump from a backup made before the breach. Watch out for weird object and column names and functions with SECURITY DEFINER.
I've been working on a backup / restore for a Postgres server for quite a while now. It's an Azure Windows Virtual Machine (Windows server 2012).
The database isn't that big (near 5Gb), but the restore takes (literally) days. I've tried (several) times with different settings to restore the database, but all of the times it took days to "finish" (it didn't finish - I killed the process because I didn't see anything happening, that's why I'm running the job verbose this time).
I've now been running the job (verbose one) for 5 days straight and still it isn't finished. It's inserting rows (or at least displaying the rows), but it's still running.
Currently I'm using this command:
pg_restore -Fc -v --jobs=2 --host=localhost [filename]
Jobs is set at 2 because it's a dual core server. Like I said: different settings still very very slow.
What is wrong - should I be "tuning" the database before the restore or what?
This is a test-server setup. When we're doing with the test the current data need to be restored (again) to the new production server: we can't afford to wait days on end before the production environment comes online.
It's not pushing errors into the logs or something - it just keeps running and running and running...
So what am I doing wrong?