I know the function "pg_dump" for backup and restore.
But I never tried because so scared.
Question is simple. Can I use that function for graph data?
Or Is the other function supported for that? There's no information in their documentation.
You may refer the postgreSQL pg_dump document coz nothing different from doing bakcup on postgreSQL.
I referred the guide to create dump script with crontab and both dump and restore worked fine.
In my case, I used pg_dump for creating dump file and restore it with psql. You may choose pg_restore instead if necessary.
agens#karl ~] pg_dump --port=5432 --username=agens --file=agens.dump agens
agens#karl ~] psql --port=5432 --username=agens --dbname=agens2 -f agens.dump
However, I no longer use pg_dump for backup task due to the incremental bakcup requirement. So I googled available backup OSS for postgresql. Among the options I searched, pg_rman is currently what I am using.
It made me easier to build a scheduling script for archive backup every 6 hours, incremental backup every day and full backup every week and those jobs are working properly more than 2 months so far.
Restoring the data on other severs is tested successfully as well.
Hope this helpful for you.
Related
Ok, I screwed up.
I dumped one of my psql (9.6.18) staging database with the following command
pg_dump -U postgres -d <dbname> > db.out
And after doing some testing, I "restored" the data using the following command.
psql -f db.out postgres
Notice the absence of -d option? yup. And that was supposed to be the username.
Annnd as the database happend to have the same name as its user, it overwrote the 'default' database (postgres), which had data that other QAs are using.
I cancelled the operation quickly as soon as I realised my mistake, but the damage was still done. Around 1/3 ~ 1/2 of the database is roughly identical to the staging database - at least in terms of the schema.
Is there any way to revert this? I am still looking for any other dumps if any of these guys made one. But I don't think there is any past two to three months. Seems like I got no choice but to own up and apologise to them in the morning.
Without a recent dump or some sort of PITR replication setup, you can't un-revert this easily. The only option is to manually go through the log of what was restored and remove/alter it in the postgres database. This will work for the schema, the data is another matter. FYI, the postgres database should not really be used as a 'working' database. It is there to be a database to connect to for doing other operations, such as CREATE DATABASE or to bootstrap your way into a cluster. If left empty then the above would not have been a problem. You could have done, from another database, DROP DATABASE postgres; and then CREATE DATABASE postgres.
Do you have a capture of the output of the psql -f db.out postgres run?
Since the pg_dump didn't specify --clean or -c, it should not have overwritten anything, just appended. And if your tables have unique or primary keys, most of the data copy operations should have failed with unique key violations and rolled back. Even one overlapping row (per table) would roll back the entire dataset for that table.
Without having the output, it will be hard to figure out what damage has actually been done.
You should also immediately copy the pg_xlog data someplace safe. If it comes down to it, you might be able to use pg_xlogdump to figure out what changes committed and what did not.
I'm using pg_dump in script which runs everyday in night to take backup of multiple DBs. we have around 20 DBs and we noticed that one DB backup takes more than 6 hrs and then got failed and after that other DBs backup started. is there anyway to fine tune this? instead to pg_dump wait for many hrs to finish that broken DB backup?
Thanks
Run the dump scripts for each database in separate scripts so they are not dependent on each other. Otherwise use the third party solutions suggested in comment.
Hello experts I am using postgres 9.5 . When I take a backup from pgadmin it has 950 MB size but when i take the same db backup from pgdump.exe command the backup size is with 7.5 GB. I am confused which backup file will be secured for me that I can use to restore? the restoring process is also slow in postgresql. Please help me.
When you backup something in pgadmin it just calls pg_dump with appropriate options, so both your backups are made by the same pg_dump utility.
I guess you're comparing dumps in two different formats.
Default format for pg_dump is plain, which is basically an enormous uncompressed SQL file.
As for pgadmin, it uses custom format by default, which is a highly compressed binary file.
Also note that pgadmin always displays the actual pg_dump command used to create your dump in the log window, along with its full output.
You should be able to call this command in your command prompt to generate an identical backup file.
You can read more about different output formats and other pg_dump options in PostgreSQL docs.
I have a site that uses PostgreSQL. All content that I provide in my site is created at a development environment (this happens because it's webcrawler content). The only information created at the production environment is information about the users.
I need to find a good way to update data stored at production. May I restore to production only the tables updated at development environment and PostgreSQL will update this records at production or the best way would be to backup the users information at production, insert them at development and restore the whole database at production?
Thank you
You can use pg_dump to export the data just from the non-user tables in the development environment and pg_restore to bring that into prod.
The -t switch will let you pick specific tables.
pg_dump -d <database_name> -t <table_name>
https://www.postgresql.org/docs/current/static/app-pgdump.html
There are many tips arounds this subject here and here.
I'd suggest you to take a look on these links before everything.
If your data is discarded at each update process then a plain dump will be enough. You can redirect pg_dump output directly to psql connected on production to avoid pg_restore step, something like below:
#Of course you must drop tables to load it again
#so it'll be reasonable to make a full backup before this
pg_dump -Fp -U user -h host_to_dev -T=user your_db | psql -U user -h host_to_production your_db
You might asking yourself "Why he's saying to drop my tables"?
Bulk loading data on a fresh table is faster than deleting old data and inserting again. A quote from the docs:
Creating an index on pre-existing data is quicker than updating it incrementally as each row is loaded.
Ps¹: If you can't connect on both environment at same time then you need to do pg_restore manually.
Ps²: I don't recommend it but you can append --clean option on pg_dump to generate DROP statements automatically. Be extreme careful with this option to avoid dropping unnexpected objects.
A linux VM with postgres 9.4 was hacked into. (Two processes taking 100% cpu, weird files in /tmp, did not reoccur after kill(s) and restart.) It was decided to install the system from scratch on a new machine (with postgres 9.6). The only data needed was in one of postgres databases. A pg_dump of the database was made after the attack.
Regardless of whether the data - the tables/rows/etc. - were modified during the attack: is it safe to restore the database in the new system?
I consider using pg_restore with the -O option (ignores the user permissions)
The two dangers are:
important data could have been modified
back doors could have been installed in your database
With the first, you're on your own how to verify that your data are ok. The safest thing would be to use a backup from before the machine was compromized, but this would mean data loss.
For the second, I would run a pg_dumpall -s and spend a day reading it carefully. Compare it with a dump from a backup made before the breach. Watch out for weird object and column names and functions with SECURITY DEFINER.