pg_restore does not work with unknown error message - postgresql

I tried to pg_dump from Postgresql 12.6 and pg_restore to Postgresql 14.5.
What I want is to move one table.
Here is the pg_dump command
pg_dump -Fc -b -U postgres -t test_eu3 -f C:\Users\...\Desktop\test_eu3.sql rulings
Here is the pg_restore command
pg_restore -v -U app_admin -d pre_rulings -t test_eu3 test_eu3.sql
Here is a message after hitting pg_restore command.
C:\Users\...>pg_restore -v -U app_admin -d pre_rulings -t test_eu3 test_eu3.sql
pg_restore: connecting to database for restore
Password:
pg_restore: implied data-only restore
pg_restore: processing data for table "public.test_eu3"
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 2915; 0 3097390 TABLE DATA test_eu3 postgres
pg_restore: error: could not execute query: ERROR: ????????????"public.test_eu3"??????????s
Command was: COPY public.test_eu3 (id, "national", item_day, item_hs_all, item_hs2, item_hs4, item_hs6, item_place, image_amount, img_name_all, item_image, other_info, org_discription, org_discription2, eng_discription, eng_discription2, id2) FROM stdin;
pg_restore: warning: errors ignored on restore: 1
C:\Users\...>
It seems pg_dump woks without any problem, but pg_restore does not work with error message of "public.test_eu3" I don't know what it's indicating "public.test_eu3"?.
I've read pg_dump and pg_restore but it seems there is no solution for this problem.
How can I restore the database?
I just mentioned the above settings in this question but still if more code is required then tell me I'll update my question with that information. Thank you
Here is a message after hitting ALTER ROLE app_admin SET lc_messages = 'C'; command and restore.
postgres=# ALTER ROLE app_admin SET lc_messages = 'C';
ALTER ROLE
postgres=# \q
C:\Users\enosh>pg_restore -v -U app_admin -d pre_rulings -t test_eu3 test_eu3.sql
pg_restore: connecting to database for restore
Password:
pg_restore: implied data-only restore
pg_restore: processing data for table "public.test_eu3"
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 2915; 0 3097390 TABLE DATA test_eu3 postgres
pg_restore: error: could not execute query: ERROR: relation "public.test_eu3" does not exist
Command was: COPY public.test_eu3 (id, "national", item_day, item_hs_all, item_hs2, item_hs4, item_hs6, item_place, image_amount, img_name_all, item_image, other_info, org_discription, org_discription2, eng_discription, eng_discription2, id2) FROM stdin;
pg_restore: warning: errors ignored on restore: 1

The parameter lc_messages is set to a language that your terminal cannot display. There are two options:
If you connect to PostgreSQL with a superuser, you can use the PGOPTIONS environment variable to override the setting during your pg_restore. On Unix-like systems, that would work as follows:
export PGOPTIONS='-c lc_messages=C'
pg_restore ...
On Windows, you can use
set PGOPTIONS="-c lc_messages=C"
pg_restore ...
This will produce English error messages.
If you don't connect to PostgreSQL with a superuser, you are not allowed to change lc_messages. In that case, your only option is to ask the database administrator to change lc_messages in postgresql.conf (and reload PostgreSQL), or to set the parameter on your user:
ALTER ROLE app_admin SET lc_messages = 'C';

Related

error during postgresql db backup restoration

A database's backup file created in Windows 7 with:
pg_dump -U postgres -Fc [db_name] >D:\[db_backup_file].sql
Then I dropped it and restored it to test the process with:
pg_restore -U postgres -C -d postgres D:\[db_backup_file].sql
Everything worked fine.
However as I tried to restore it in Ubuntu 20.04 in a different device, I got an error:
could not execute query: ERROR: invalid locale name: (same as here)
So I followed the given instructions creating the database,
sudo -u postgres psql
create database [db_name];
and then I placed in the terminal the following command to restore backup:
pg_restore -U postgres -d postgres /home/../../[db_backup_file].sql
But again I got errors, as many were the tables, multiplied by four.
So for every table I get the following errors:
pg_restore: from TOC entry 315; 1259 29971 TABLE [table_name] postgres
pg_restore: error: could not execute query: ERROR: relation [table_name] already exists
Command was: CREATE TABLE public.[table_name] (
[pkey_column_name] integer NOT NULL,
.......
.......
.......
.......
.......
.......
);
pg_restore: from TOC entry 314; 1259 29969 SEQUENCE [table_name]_[pkey_column_name]_seq postgres
pg_restore: error: could not execute query: ERROR: relation "[table_name]_[pkey_column_name]_seq" already
exists
Command was: CREATE SEQUENCE public.[table_name]_[pkey_column_name]_seq
AS integer
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;
pg_restore: from TOC entry 3522; 0 29971 TABLE DATA [table_name] postgres
pg_restore: error: COPY failed for table "[table_name]": ERROR: duplicate key value violates unique constraint "[table_name]_pkey"
DETAIL: Key ([pkey_column_name])=(1) already exists.
CONTEXT: COPY [table_name], line 1
pg_restore: from TOC entry 3267; 2606 29976 CONSTRAINT [table_name] [table_name]_pkey postgres
pg_restore: error:
could not execute query: ERROR: multiple primary keys for table "[table_name]" are not allowed
Command was: ALTER TABLE ONLY public.[table_name]
ADD CONSTRAINT [table_name]_pkey PRIMARY KEY ([pkey_column_name]);
When the tables were created the primary keys - if something has to do with it - were defined as auto increment, in the form of:
CREATE TABLE [table_name] (
[pkey_column_name] serial primary key,
.......
.......
.......
.......
.......
.......
);
Can anyone please help me about that?
EDIT: Actually the missing codepage type in the first error in my post yesterday was "Greek_Greece.1253". I used the locale -a command as you said, and I saw that among others my Ubuntu OS has en_US.UTF-8 and el_GR.UTF-8. So I'm wondering if the issue could be that incompatibility between Windows and Ubuntu character sets. If yes, how you thing that I could manage it?
Luckily the windows 7 device from which the backup files come from is still in use, and so the databases are active. But what I tried to create again the databases giving for LC_COLLATE and LC_CTYPE values compatible with ubuntu, didn't work.
EDIT 2: Finally it was that windows-linux incompatibility in characters encoding.
When I tried to use en_US.UTF-8 or el_GR.UTF-8 with encoding parameter as following:
pg_dump -E en_US.UTF-8 -U postgres -Fc [db_name] > D:\[backup_file].sql
I was getting:
pg_dump: invalid client encoding "en_US.UTF-8" specified
Then I tried to create in ubuntu the db before I restore it, under the command:
CREATE DATABASE database_name WITH ENCODING 'utf8' LC_COLLATE='el_GR.utf8' LC_CTYPE='el_GR.utf8' TEMPLATE template0;
and then:
pg_restore -U postgres -d postgres ~/../../backup_file.sql
but I got the same batch of errors I had in the initial post.
So the solution was to create a new database in windows, but now under 'C' char encoding (POSIX wouldn't be accepted), copy the tables from one database to another:
pg_dump -U postgres -t [table_name] [database_name] | psql -U postgres -d [database_name]
and then dump the newly created db, and restore it in ubuntu environment.
It could be that your Ubuntu OS does not have an en_US.UTF-8 locale. You can check for this by using this command in terminal:
locale -a # list all locales known to OS
If you cannot find the locale in the list, try making a new one according to this post
EDIT
With the additional information that the Windows encoding is Greek_Greece.1253, it still sounds like there is a mismatch. According to the pg_dump docs, you can explicitly set the encoding using the -E option. Probably you want to set it to something that Ubuntu can handle (i.e. en_US.UTF-8 or el_GR.UTF-8)
-E encoding
--encoding=encoding
Create the dump in the specified character set encoding. By default, the dump is
created in the database encoding. (Another way to get the same result is to set the
PGCLIENTENCODING environment variable to the desired dump encoding.)

pg_dump/pg_restore lc_collate English_United States.1252 Windows Linux migration

I want to use
pg_dump -j 8 -Fd -f /tmp/newout.dir fsdcm_external
without dumping LC_COLLATE ('English_United States.1252'), as it is incompatible with my Linux box.
I initially successfully migrated a DB with LC_COLLATE = 'English_United States.1252' with the following procedure:
#Windows machine:
pg_dump postgres > "/drive/folder/options.bak"
#Linux machine:
psql postgres < "/drive/folder/backup.bak"
However, I prefer to use "jobs" for parallel dumping and restoring (as in A faster way to copy a postgresql database (or the best way)), therefore I tried:
#Windows machine:
pg_dump -j 8 -Fd -f /tmp/newout.dir fsdcm_external
#Linux machine:
pg_restore -j 8 --format=d -C -d postgres /tmp/newout.dir/
I expected to have the same LC_COLLATE neutral dump, but instead I received this error:
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 3498; 1262
12401 DATABASE postgres postgres
pg_restore: [archiver (db)] could not execute query: ERROR:
invalid locale name: "English_United States.1252"
Command was: CREATE DATABASE postgres WITH TEMPLATE = template0
ENCODING = 'UTF8' LC_COLLATE = 'English_United States.1252'
LC_CTYPE = 'E...
How do I specify in my pg_dump or pg_restore to refrain from replicating LC_COLLATE settings? Can I only dump the tables, data and indexes for example?
Your problem is the -C option of pg_restore.
Instead, first create the database, and then run pg_restore without -C:
createdb fsdcm_external
pg_restore -j 8 --format=d -d fsdcm_external /tmp/newout.dir/
I spent last 3 hours trying to restore with the PGAdmin GUI but it wouldn't let me. Thank goodness for the command line. Thanks Laurenz.
For anyone else reading, run it in the Terminal (Mac). Not in pysql. You may to need to indicate -U postgres the default user.
pg_restore -U postgres --exit-on-error --verbose --dbname=yourdbname /Users/gc/Downloads/file.tar

pg_restore: using the -L (use-list) option not working

I every now and then restores a db-dump. To save time and I want to exclude some large un-needed tables.
I try to use the -L (use-list) option but when running the pg_restor command it does not work.
I get the same result even if I'm using a un-edited list.
This is the command I use:
pg_restore -v -U [user] -L "restore.pgdump.list" -d [dbname]-h localhost -p 5432 [dump-file]
This is the result:
pg_restore: [archiver] WARNING: line ignored: ÿþ
pg_restore: connecting to database for restore
Password:
pg_restore: implied data-only restore
If I go with the same command without the L-option the database is restored correctly.
Would much appreciate your help
Peter

pg_restore certain tables only

I'm trying to write a pg_restore command to restore only certain tables (and their data) to my database.
Note: every command described begins with me dropping and re-creating the database and ends in: -v -x -O -j 8 -h localhost -U username -d database file.dump (For the curious, I didn't want to use --clean because the database that the dump came from has a different name.)
Since pg_restore works fine for me (with the above args), I looked at the pg_restore documentation, and tried something like this:
pg_restore -t table1 -t table2 ... (there are 121 tables I specify in this way).
However, I get errors like the following:
pg_restore: creating TABLE people
pg_restore: [archiver (db)] Error from TOC entry 123; 1234 12345 TABLE people dumped_table_username
pg_restore: [archiver (db)] could not execute query: ERROR: type "hstore" does not exist
LINE 14: extra_data hstore,
^
Command was: CREATE TABLE people (
id integer NOT NULL,
name string,
age integer,
date_of_birt...
I don't see why this would be an issue only when the -t flag is set, but it appears to be.
What's going on?
Edit: looks like this is a duplicate of pg_restore on table failing because of hstore, which was recently asked and has no accepted answer as of this time.
Apparently, pg_restore with the -t/--table flag set doesn't run CREATE EXTENSION commands that are in the dump file (because they're not technically part of that table). My problem was solved by manually running psql database -c "CREATE EXTENSION hstore;" before the pg_restore command.

pg_restore toc error

i was using the following syntax for pg_dump and restore
pg_dump eval --inserts -b -c --encoding UTF8 -Fc -f eval.sql.tar.gz -x -U postgres
createdb -T template0 test -U postgres
pg_restore -d test eval.sql.tar.gz -e -U postgres
the dump was successfull with no errors, but restore makes a some errors, i am dumping and restoring in same machine with same user and privilege all...
i have tried out with other formats also, plain, tar, compressed all gets the same error..
my version of pg is 8.4.11 and psql version is 8.4.11
i am not sure what makes these errors.. can anyone help me
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 4965; 0 138871 TABLE DATA ir_act_report_xml insigni
pg_restore: [archiver (db)] could not execute query: ERROR: invalid input syntax for integer: "purchase.order"
LINE 1: ...st for Quotation', 'ir.actions.report.xml', NULL, 'purchase....
^
Command was: INSERT INTO ir_act_report_xml VALUES (350, 'Request for Quotation', 'ir.actions.report.xml', NULL, 'purchase.order', 'purcha...
this did the trick
pg_dump database_name -c -Ft -f file_name.tar
pg_restore -d database_name -c file_name.tar
before this i was trying to restore with out including -c(clean)
even though -c is included in pg_dump it is not used in pg_restore unless we say to use...
The solution in my case:
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U username -d database_name dump_name.dump
This worked for me:
Increase the max_wal_size postgresql setting (max_wal_size = 2GB) in postgresql.conf