I have a Postgres database in production environment, and it has millions of records in tables. So I wanted to take a backup using pg_dump for some investigation.
But this database is so busy. So I am afraid if backup operation is caused any server issue like slow down server or crash database etc. as it is busy database.
Can anyone share if there is any risk? And please give some idea about best practice to take a backup from Postgres with no risk.
Running pg_dump will not cause a server crash, but it will add some extra CPU and particularly I/O load. You can test if that is a problem, pg_dump can be canceled any time.
On a busy database, it can also lead to table bloat, because old row versions have to be retained for the duration of pg_dump and cannot be vacuumed.
There are some alternatives:
Run pg_dump against a standby server.
Use pg_basebackup to perform a physical backup. That can be throttled to reduce the I/O load.
Related
I have a chron job that runs on a stateless server. On this chron job, I am trying to take a snapshot of my Postgres GCP Sql db (PRODUCTION_DATABASE), save it to S3 and then upload it to my staging, qa-1, dev databases. The problem is one table, call it LARGE_TABLE, needs to be shrunk because the table size is growing rapidly, thus causing problems and exceeding timeouts. Does anyone have any advice on how to get this done?
I tried running the cloud_sql_proxy to run pg_dump but no go with that method. Is there a way I can truncate one table and make a backup?
I'm attempting to upgrade heroku PostgreSQL instances from pg11 to pg12 using the copy method as my testing environments are on hobby instances. At the end of the process it appears to be hanging for a long time (does not exit after >30 minutes for a 120MB database). The datastore view suggests everything is fine, I have the same number of rows, but there are issues.
It appears to be the fault of a materialized view. If I connect to the database and look through the tables and views, only one appears to be empty. Using postico, it waits and waits for the view's structure, but doesn't give the usual warning for an unpopulated view.
I can recreate the stalling behaviour by creating a local pg12 database and attempting to use pg_restore with a recent backup. Along the same lines, I appear to be able to get it working by creating an empty local database, running all the db migrations, truncating all tables and sequences, and then doing a --data-only --disable-triggers load from the same backup. Not a particularly smooth or inspiring migration plan plan. Using --verbose doesn't show up any obvious errors, the last thing I get is that it's creating the problematic materialized view.
I've also set log_statement to all, and the last one I get is that it's refreshing the problematic view. At this point, the postgres command starts using ~100% CPU.
Locally, I'm using this command to restore:
pg_restore --verbose --clean --no-acl --no-owner -h localhost -d database_name database_backup.dump
This is the command we use regularly to restore production backups for local development.
Are there any known gotchas with upgrading from 11 to 12, or ways that I might be able to extract more information about what's going on?
It has probably chosen an appalling plan for doing the materialized view query, due to lack of statistics at the time the refresh was launched.
You could kill the process, then restart the refresh once stats are gathered (which they might already be.)
If starting from scratch, you could run pg_restore with --section of pre-data and data, then do an ANALYZE, then do post-data.
Heroku warns of using Multiple Schemas with Postgres. But does not specify numerous operational problems caused.
As posted on Heroku docs:
The most common use case for using multiple schemas in a database is
building a software-as-a-service application wherein each customer has
their own schema. While this technique seems compelling, we strongly
recommend against it as it has caused numerous cases of operational
problems. For instance, even a moderate number of schemas (> 50) can
severely impact the performance of Heroku’s database snapshots tool,
PG Backups.
I think, the problem of Backups can be solved by adding a follower db.
I have 60 tables per schema, so with 1000 schemas I will have 60,000 tables. How will this impact database performance? What kind of problems I can expect while scaling?
The first issues with running a large volume of schemas and/or tables don't typically interfere with a running database. The major problem that operators encounter is that they will be unable to create logical backups of the database. Running heroku pg:backups is likely to fail as is running pg_dump manually. Typically, this is the error you'll see in the attempted backup logs:
ERROR: out of shared memory HINT: You might need to increase max_locks_per_transaction
The large volume of locks needed end up causing OOM conditions for the database. This isn't always a problem on Heroku. If you're using a production database you can rely on their point-in-time recovery option as your DR solution. That being said, exporting the data off Heroku will be difficult if you can't run a logical backup since they don't currently support external replication. It's less than ideal but hypothetically you could try to dump the database schema by schema in order avoid the OOM conditions.
Is it possible to access, particularly insert into, a database while it's being restored by pg_restore from a custom format.
If yes, then should I care about preventing clients from accessing the database while pg_restore is running, or the restore operation is "transactional" so that after it ends all changes made by clients since its start will be lost?
If you want to keep concurrent sessions out of your database during pg_restore, you'll have to block them with a pg_hba.conf entry.
There is no protection from concurrent sessions inserting or otherwise modifying data while pg_restore is running.
We're serving a Django/Postgres site running on a VM hypervisor. We're now trying to figure out our back up strategy and have two probable options:
Back up the DB directly using pg_dump
Back up the VM directly by copying the VM image
I'm with the latter as I think, I could simply back up everything that has to do with the site. I'm not sure whether I have to shut down the VM for this though.
What is a better and more recommended way of backing up a DB? Are there any reasons for not using the VM backup?
Thanks
The question basically boils down to, can you consider a hot copy of PostgreSQL's data files a backup?
The answer is: not really. PostgreSQL tries very hard through the use of WAL to ensure that its files are in a consistent state all the time and that it can survive a power failure, but starting it up from a copy of these files puts PostgreSQL into recovery mode. If the backup happened at the wrong second and PostgreSQL can't recover from the state of these files, your backup is useless. You don't want your backup/restore mechanism to depend on the recovery mechanism (unless you're dealing with "crash only" software, which PostgreSQL is not).
The probability of PostgreSQL not being able to recover from these files is not high, but it's not zero either. The probability of PostgreSQL not being able to load an SQL dump that it made, on the other hand, is zero. I prefer backup choices with lower probabilities of failure. pg_dump was designed for doing backups.
PostgreSQL recommends using pg_dump for backups, as a file system (or VM) backup requires the database to be shut down (and has other drawbacks):
http://www.postgresql.org/docs/8.1/static/backup-file.html
Edit: Also, a pg_dump backup will be significantly smaller than a filesystem dump of the same database.
There is an additional option. With PostgreSQL you can make an online backup that allows you to snapshot the file system and maintain consistency. You can see details here:
http://www.postgresql.org/docs/9.0/static/continuous-archiving.html
We use this exact method for making backups when we run PostgreSQL in a VM.