i was using the following syntax for pg_dump and restore
pg_dump eval --inserts -b -c --encoding UTF8 -Fc -f eval.sql.tar.gz -x -U postgres
createdb -T template0 test -U postgres
pg_restore -d test eval.sql.tar.gz -e -U postgres
the dump was successfull with no errors, but restore makes a some errors, i am dumping and restoring in same machine with same user and privilege all...
i have tried out with other formats also, plain, tar, compressed all gets the same error..
my version of pg is 8.4.11 and psql version is 8.4.11
i am not sure what makes these errors.. can anyone help me
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 4965; 0 138871 TABLE DATA ir_act_report_xml insigni
pg_restore: [archiver (db)] could not execute query: ERROR: invalid input syntax for integer: "purchase.order"
LINE 1: ...st for Quotation', 'ir.actions.report.xml', NULL, 'purchase....
^
Command was: INSERT INTO ir_act_report_xml VALUES (350, 'Request for Quotation', 'ir.actions.report.xml', NULL, 'purchase.order', 'purcha...
this did the trick
pg_dump database_name -c -Ft -f file_name.tar
pg_restore -d database_name -c file_name.tar
before this i was trying to restore with out including -c(clean)
even though -c is included in pg_dump it is not used in pg_restore unless we say to use...
The solution in my case:
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U username -d database_name dump_name.dump
This worked for me:
Increase the max_wal_size postgresql setting (max_wal_size = 2GB) in postgresql.conf
Related
I am currently working on migrating a postgres database between two azure postgres instances. Since these are managed services, I am not able to use a superuser-account (those are not provided). I am first migrating my roles, then my database, with the following commands:
# https://learn.microsoft.com/en-us/azure/postgresql/single-server/how-to-upgrade-using-dump-and-restore#migrate-the-roles
# Migrate the roles
pg_dumpall -r --host=$OLD_HOST --username=$OLD_USERNAME --database=$DB_NAME > roles.sql
sed -i 's/NOSUPERUSER//g' roles.sql
sed -i 's/NOBYPASSRLS//g' roles.sql
psql -f roles.sql --host=$NEW_HOST --username=$NEW_USERNAME --dbname=postgres --clean --if-exists
pg_dump -Z0 -Fd -j 12 -h $OLD_HOST -U $OLD_USERNAME -d $DB_NAME -f $DIRECTORY
pg_restore -h $NEW_HOST -U $NEW_USERNAME -j 12 -Fd -C -d $DB_NAME $DIRECTORY
Unfortunately this results in a couple of errors like these:
pg_restore: from TOC entry 720; 1259 1421377 SEQUENCE id_publicatie_seq svp
pg_restore: error: could not execute query: ERROR: permission denied for schema svp
Command was: ALTER TABLE svp.id_publicatie_seq OWNER TO svp;
pg_restore: from TOC entry 718; 1259 1421365 TABLE publicatie svp
pg_restore: error: could not execute query: ERROR: permission denied for schema svp
Command was: ALTER TABLE svp.publicatie OWNER TO svp;
I'm able to get rid of these errors by adding the --no-owner flag in my pg_restore command, but this is a problem since I specifically want to keep all owners exactly the same as in my original database. Am I overseeing something?
I tried to pg_dump from Postgresql 12.6 and pg_restore to Postgresql 14.5.
What I want is to move one table.
Here is the pg_dump command
pg_dump -Fc -b -U postgres -t test_eu3 -f C:\Users\...\Desktop\test_eu3.sql rulings
Here is the pg_restore command
pg_restore -v -U app_admin -d pre_rulings -t test_eu3 test_eu3.sql
Here is a message after hitting pg_restore command.
C:\Users\...>pg_restore -v -U app_admin -d pre_rulings -t test_eu3 test_eu3.sql
pg_restore: connecting to database for restore
Password:
pg_restore: implied data-only restore
pg_restore: processing data for table "public.test_eu3"
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 2915; 0 3097390 TABLE DATA test_eu3 postgres
pg_restore: error: could not execute query: ERROR: ????????????"public.test_eu3"??????????s
Command was: COPY public.test_eu3 (id, "national", item_day, item_hs_all, item_hs2, item_hs4, item_hs6, item_place, image_amount, img_name_all, item_image, other_info, org_discription, org_discription2, eng_discription, eng_discription2, id2) FROM stdin;
pg_restore: warning: errors ignored on restore: 1
C:\Users\...>
It seems pg_dump woks without any problem, but pg_restore does not work with error message of "public.test_eu3" I don't know what it's indicating "public.test_eu3"?.
I've read pg_dump and pg_restore but it seems there is no solution for this problem.
How can I restore the database?
I just mentioned the above settings in this question but still if more code is required then tell me I'll update my question with that information. Thank you
Here is a message after hitting ALTER ROLE app_admin SET lc_messages = 'C'; command and restore.
postgres=# ALTER ROLE app_admin SET lc_messages = 'C';
ALTER ROLE
postgres=# \q
C:\Users\enosh>pg_restore -v -U app_admin -d pre_rulings -t test_eu3 test_eu3.sql
pg_restore: connecting to database for restore
Password:
pg_restore: implied data-only restore
pg_restore: processing data for table "public.test_eu3"
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 2915; 0 3097390 TABLE DATA test_eu3 postgres
pg_restore: error: could not execute query: ERROR: relation "public.test_eu3" does not exist
Command was: COPY public.test_eu3 (id, "national", item_day, item_hs_all, item_hs2, item_hs4, item_hs6, item_place, image_amount, img_name_all, item_image, other_info, org_discription, org_discription2, eng_discription, eng_discription2, id2) FROM stdin;
pg_restore: warning: errors ignored on restore: 1
The parameter lc_messages is set to a language that your terminal cannot display. There are two options:
If you connect to PostgreSQL with a superuser, you can use the PGOPTIONS environment variable to override the setting during your pg_restore. On Unix-like systems, that would work as follows:
export PGOPTIONS='-c lc_messages=C'
pg_restore ...
On Windows, you can use
set PGOPTIONS="-c lc_messages=C"
pg_restore ...
This will produce English error messages.
If you don't connect to PostgreSQL with a superuser, you are not allowed to change lc_messages. In that case, your only option is to ask the database administrator to change lc_messages in postgresql.conf (and reload PostgreSQL), or to set the parameter on your user:
ALTER ROLE app_admin SET lc_messages = 'C';
I want to use
pg_dump -j 8 -Fd -f /tmp/newout.dir fsdcm_external
without dumping LC_COLLATE ('English_United States.1252'), as it is incompatible with my Linux box.
I initially successfully migrated a DB with LC_COLLATE = 'English_United States.1252' with the following procedure:
#Windows machine:
pg_dump postgres > "/drive/folder/options.bak"
#Linux machine:
psql postgres < "/drive/folder/backup.bak"
However, I prefer to use "jobs" for parallel dumping and restoring (as in A faster way to copy a postgresql database (or the best way)), therefore I tried:
#Windows machine:
pg_dump -j 8 -Fd -f /tmp/newout.dir fsdcm_external
#Linux machine:
pg_restore -j 8 --format=d -C -d postgres /tmp/newout.dir/
I expected to have the same LC_COLLATE neutral dump, but instead I received this error:
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 3498; 1262
12401 DATABASE postgres postgres
pg_restore: [archiver (db)] could not execute query: ERROR:
invalid locale name: "English_United States.1252"
Command was: CREATE DATABASE postgres WITH TEMPLATE = template0
ENCODING = 'UTF8' LC_COLLATE = 'English_United States.1252'
LC_CTYPE = 'E...
How do I specify in my pg_dump or pg_restore to refrain from replicating LC_COLLATE settings? Can I only dump the tables, data and indexes for example?
Your problem is the -C option of pg_restore.
Instead, first create the database, and then run pg_restore without -C:
createdb fsdcm_external
pg_restore -j 8 --format=d -d fsdcm_external /tmp/newout.dir/
I spent last 3 hours trying to restore with the PGAdmin GUI but it wouldn't let me. Thank goodness for the command line. Thanks Laurenz.
For anyone else reading, run it in the Terminal (Mac). Not in pysql. You may to need to indicate -U postgres the default user.
pg_restore -U postgres --exit-on-error --verbose --dbname=yourdbname /Users/gc/Downloads/file.tar
I have a script where pg_dump is failing with a message like this:
pg_dump -h db1 --format plain --encoding UTF8 --schema=public --schema-only --no-owner me
pg_dump: [archiver (db)] query failed: ERROR: permission denied for relation notmytable
pg_dump: [archiver (db)] query was: LOCK TABLE public.notmytable IN ACCESS SHARE MODE
This is causing the whole dump to abort.
Is there a way to either:
Ignore tables that aren't owned by our user?
Ignore errors?
I really don't want these tables in the dump, so even if we could get access to them, that wouldn't exactly solve the problem.
(Postgres 9.6.3)
It doesn't appear there is a standard way to do this, but using the --exclude-table flag, we can use a workaround:
export EXCLUDETABLE=$(psql -t -h $HOST -d $DBNAME -c "select '--exclude-table=' || string_agg(tablename,' --exclude-table=') FROM pg_catalog.pg_tables WHERE tableowner NOT LIKE 'myuser';" )
This sets EXCLUDETABLE to look like --exclude-table=foo --exclude-table=blah
Now we pass that to pg_dump:
echo Excluding these tables from dump: $EXCLUDETABLE
pg_dump -h $HOST --format plain --encoding UTF8 --schema=public --schema-only --no-owner $EXCLUDETABLE $DBNAME > public-schema.sql
I downloaded the xbrldb_SEC_pg_2014-11-02.pg.gzip postgres pg_dump file from arelle.org. I then ran the schema ddl file in pgAdminIII and it recreated all of the databases, functions, etc.
When I try to restore the databases using the following:
desktop:~/Downloads$ sudo postgres zcat xbrldb_SEC_pg_2014-11-02.pg.gzip | psql -U postgres public
I get:
sudo: postgres: command not found psql: FATAL: Peer authentication failed for user "postgres"
I can zcat the file into a file to expand it. Looks like it is a pg_dump file.
postgres=> pg_restore -a /home/jeremy/Downloads/xbrldb_SEC_pg_2014-11-02.txt
postgres-> ;
ERROR: syntax error at or near "pg_restore"
LINE 1: pg_restore -a /home/jeremy/Downloads/xbrldb_SEC_pg_2014-11-0...
^
postgres=> pg_restore -a postgres /home/jeremy/Downloads/xbrldb_SEC_pg_2014-11-02.txt;
ERROR: syntax error at or near "pg_restore"
LINE 1: pg_restore -a postgres /home/jeremy/Downloads/xbrldb_SEC_pg_...
So then I tried to use PG Admin III, and my output:
/usr/bin/pg_restore --host localhost --port 5432 --username "postgres" --dbname "public" --role "postgres" --no-password --section data --data-only --exit-on-error --table accession --schema public --verbose "/home/jeremy/Downloads/xbrldb_SEC_pg_2014-11-02.backup"
pg_restore: [archiver] input file appears to be a text format dump. Please use psql.
Process returned exit code 1.
May I please ask what I need to do to get the databases restored?
Does anyone know what I need to do to get the database updated from 2014-11-02 to the current date?
You should run psql as postgres user, not zcat, so try to use following:
zcat xbrldb_SEC_pg_2014-11-02.pg.gzip | sudo -u postgres psql public
PS pg_restore is an utility, not a PostgreSQL command, that means you should run it from command line, not from psql.