I Tried to take a dump of my database. While Dumping i received the below error. I knew a table in my database is corrupted and i can use exclude option to exclude the corrupted table and take dump of the remaining tables. But I am searching for an option to dump even the corrupted table excluding the invalid page blocks.
The command i used to take dump is :
pg_dump -U postgres -p 5432 -d DBNAME -v -n SCHEMA_NAME -f "OUTFILE.sql"
The error prints:
pg_dump: Dumping the contents of table "TABLE_NAME" failed: PQgetResult() failed.
pg_dump: Error message from server: ERROR: invalid page in block 2145280 of relation base/16384/25464
I was able to access the corrupted table and its data. So i believe the part of the data which i can access can be dumped. Kindly share if there is any possibility to achieve this.
You could set zero_damaged_pages = on, then PostgreSQL should zero (and ignore) such pages.
Related
I try dumping tables from a production environment to a dev one. However, when dumping and restoring this table, using the following command:
pg_restore --no-owner --no-acl --clean --if-exists -d database dump_file.dump
I get an error stating that I can't drop that table unless I use something like CASCADE (i.e. dropping all other tables that depend on that one). Is there a way to determine the tables to be dropped? is there a way of maybe state in the pg_dump command to dump the table I'm looking to dump and all related tables ?
Here's the error raised:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 4066; 2606 30526 CONSTRAINT table1 pkey user
pg_restore: error: could not execute query: ERROR: cannot drop constraint pkey on table public.table1 because other objects depend on it
DETAIL: constraint id_fkey on table public.dag depends on index public.pkey
constraint id_fkey on table public.dag depends on index public.pkey
HINT: Use DROP ... CASCADE to drop the dependent objects too...
You have a table on the dev database that has a pkey that is dependent and therefore can not be dropped before the restore. This is proper behavior.
I am not seeing dumping/restoring a particular table. You are dumping/restoring the entire database.
If you want recreate the production database as a dev one then do:
pg_restore -C --no-owner --no-acl --clean --if-exists -d postgres dump_file.dump
The -C with --clean will DROP DATABASE db_name and then rebuild it from scratch by connecting to the database postgres to do the DROP/CREATE db_name and then connect to db_name to load the rest of the objects.
This is the best way to clean out cruft and start at a consistent state.
UPDATE
Update your question with the pg_dump command so it is evident what you are doing.
If you want to see whether a particular table has dependencies, in the original database use psql and do \d the_table to see what the dependencies to and from the table are. If you tell pg_dump to dump a single table it will dump just that table. It will not follow dependencies and dump those also. That is up to you to do.
Look into using a schema management tool to do your changes/migrations. I use Sqitch for this.
I dumped the database with:
pg_dump dsmartgpp -f /data1/master/backup/db.sql
and I copied the database with
nohup psql -h gpnm1 -d dsmartgpp_1 -U gpadmin < /data1/master/backup/db.sql
the log information showed some errors about the function and datatype of postgis, such as
ALTER FUNCTION
ERROR: function "gidx_out" already exists with same argument types.
ALTER FUNCTION
ERROR: type "gidx" already exists
some tables between the two database have the same number or records, but about 1MB difference in size.
That is as expected. Restoring a pg_dump creates a logical copy of the database. In particular, the copy won't be bloated (don't worry – a bit of bloat is healthy).
If you want to avoid these errors, create the new database (dsmartgpp_1) with template0 as template (empty database). You seem to have PostGIS installed in template1, which gets copied into every new database. Or you installed PostGIS in your new database, before importing the dump.
Either way, create the empty database (dsmartgpp_1) and let the dump install the PostGIS functions.
Oh, one more thing, you can use "-f /data1/master/backup/db.sql" instead of the "<" redirect.
If you want to be super careful, also add "-v ON_ERROR_STOP=1". This will stop at the first error.
I'm having trouble performing the restore from a dump. The scenario is as follows: I am migrating an environment from GCP to AWS, and at the moment I am working on the migration of the bank.
A partner dumped db that is in GCP and placed the file on AWS S3 (I don't know the command he used to perform the dump).
I created an EC2 in the AWS environment and copied the dump from S3 to EC2 (the file is 13 GB). I also created the RDS to host the new db with all the correct security group settings.
Here comes the problem, I connect to the RDS from the server (EC2) without problems, but when doing the restore using pg_restore I get the following error message: pg_restore: too many command line arguments (first is "dbclient. dump ").
The complete command I used was this:
pg_restore -h client-aurora-cluster-hmg-legado-instance-1.c23ltjbbz7ms.us-east-1.rds.amazonaws.com -U postgres -d db_hmg_legado dbclient.dump -W
OK, I changed the approach. I tried with psql instead of pg_restore and then the command was like this:
psql -h client-aurora-cluster-hmg-legado-instance-1.c23ltjbbz7ms.us-east-1.rds.amazonaws.com -U postgres -d db_hmg_legado dbclient.dump -W
Only this time it worked !!!!
But I received some error messages while performing the restore. Which I put below:
psql: dbclient.dump: 23: ERROR: schema "dw" already exists
CREATE EXTENSION
psql: dbclient.dump: 37: ERROR: must be owner of extension hstore
CREATE EXTENSION
psql: dbclient.dump: 51: ERROR: must be owner of extension intarray
CREATE EXTENSION
psql: dbclient.dump: 65: ERROR: must be owner of extension pg_trgm
CREATE EXTENSION
psql: dbclient.dump: 79: ERROR: must be owner of extension unaccent
But the restore takes a long time and is partially finished.
In general I wanted to understand why pg_restore didn't work. Has anyone ever experienced this?
And about these owner errors does anyone know how to resolve this using psql?
As documented in the manual the file to be restored is the last parameter and it is specified without a "switch". But you are using -W after the dump file. Move the -W parameter somewhere before that (although it's usually not necessary to begin with)
So you need something like this:
pg_restore -W -h ... -U postgres -d db_hmg_legado dbclient.dump
However, if the restore worked when using psql then the dump file is a "plain text" dump which can't be restored using pg_restore to begin with.
Concerning the errors:
You should restore the dump into an empty database that doesn't contain any schemas except the default ones.
You need a superuser for CREATE EXTENSION, which you don't have in a hosted database. So pre-install these extensions with the techniques that Amazon provides, then restore the dump and ignore the errors.
I was trying to upgrade Sentry and a table in my database got corrupt.
After reading about vacuum and reindex I was able to track down the issue to a single table.
Doing a select * from any other table works just fine, but this particular one seems to be problematic. Is there a way I can fix the table, or, worst case scenario, dump all other tables somehow?
pg_dump -T corrupt_table > bkp.sql doesn't work:
bash-4.4# pg_dump -U XXXXXX -T sentry_identityprovider sentry > bkp.sql
pg_dump: [archiver (db)] query failed: ERROR: cache lookup failed for attribute 1 of relation 45941
pg_dump: [archiver (db)] query was: SELECT tableoid, oid, conname, confrelid, pg_catalog.pg_get_constraintdef(oid) AS condef FROM pg_catalog.pg_constraint WHERE conrelid = '45954'::pg_catalog.oid AND contype = 'f'
Please avoid comments like "Well, go get your backups". I'm asking because I don't have a backup.
Also, please avoid comments like "Well, if you don't have backups, shit happens". I'm asking because there was an error in the execution of the backups and none were made.
Also, please avoid any other helpless comments related to backups. Really. You're not helping me that way.
At some stage I have been able to dump and restore an individual table as per below. Note in help for pg_dump that you should be able to do a full dump and exclude the corrupt table, never tried it. Not sure why yours fails, dumping a single good table might give the answer. Hope it works for you.
pg_dump -t good_table old_DB -U youruser -f good_table_BUP.sql
psql -f good_table_BUP.sql new_DB
-t, --table=TABLE dump the named table(s) only
-T, --exclude-table=TABLE do NOT dump the named table(s)
When trying to make a postgreSQL database dump we got the following error and the process stops immediately.
Command used:
openbravo#master.akluck.com:~
07/26 11:48:11> pg_dump -U tad -h localhost -p 5932 -F c -b -v -f /home/openbravo/dump26072018.dmp openbravo
Output:
pg_dump: reading schemas
pg_dump: reading user-defined tables
pg_dump: schema with OID 67046 does not exist
pg_dump: *** aborted because of error
Can anyone guide me how to sort this issue?
Update:
I followed this tutorial
http://www.aukema.org/2011/06/fixing-complex-corruption-in-my-dna.html
And I can see there are objects without a schemaname in the pg_tables.
But I don't know how to update those missing schemanames in the pg_tables. The last part of the tutorial is not quite explanatory. Hope someone can throw some light.
Finally found a way to take the backup by excluding the corrupted table as follows
pg_dump --exclude-table=ad_context_info -h localhost -p 5932 -U postgres > dumpsabnew.dmp