I'm trying to write a pg_restore command to restore only certain tables (and their data) to my database.
Note: every command described begins with me dropping and re-creating the database and ends in: -v -x -O -j 8 -h localhost -U username -d database file.dump (For the curious, I didn't want to use --clean because the database that the dump came from has a different name.)
Since pg_restore works fine for me (with the above args), I looked at the pg_restore documentation, and tried something like this:
pg_restore -t table1 -t table2 ... (there are 121 tables I specify in this way).
However, I get errors like the following:
pg_restore: creating TABLE people
pg_restore: [archiver (db)] Error from TOC entry 123; 1234 12345 TABLE people dumped_table_username
pg_restore: [archiver (db)] could not execute query: ERROR: type "hstore" does not exist
LINE 14: extra_data hstore,
^
Command was: CREATE TABLE people (
id integer NOT NULL,
name string,
age integer,
date_of_birt...
I don't see why this would be an issue only when the -t flag is set, but it appears to be.
What's going on?
Edit: looks like this is a duplicate of pg_restore on table failing because of hstore, which was recently asked and has no accepted answer as of this time.
Apparently, pg_restore with the -t/--table flag set doesn't run CREATE EXTENSION commands that are in the dump file (because they're not technically part of that table). My problem was solved by manually running psql database -c "CREATE EXTENSION hstore;" before the pg_restore command.
Related
I tried to pg_dump from Postgresql 12.6 and pg_restore to Postgresql 14.5.
What I want is to move one table.
Here is the pg_dump command
pg_dump -Fc -b -U postgres -t test_eu3 -f C:\Users\...\Desktop\test_eu3.sql rulings
Here is the pg_restore command
pg_restore -v -U app_admin -d pre_rulings -t test_eu3 test_eu3.sql
Here is a message after hitting pg_restore command.
C:\Users\...>pg_restore -v -U app_admin -d pre_rulings -t test_eu3 test_eu3.sql
pg_restore: connecting to database for restore
Password:
pg_restore: implied data-only restore
pg_restore: processing data for table "public.test_eu3"
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 2915; 0 3097390 TABLE DATA test_eu3 postgres
pg_restore: error: could not execute query: ERROR: ????????????"public.test_eu3"??????????s
Command was: COPY public.test_eu3 (id, "national", item_day, item_hs_all, item_hs2, item_hs4, item_hs6, item_place, image_amount, img_name_all, item_image, other_info, org_discription, org_discription2, eng_discription, eng_discription2, id2) FROM stdin;
pg_restore: warning: errors ignored on restore: 1
C:\Users\...>
It seems pg_dump woks without any problem, but pg_restore does not work with error message of "public.test_eu3" I don't know what it's indicating "public.test_eu3"?.
I've read pg_dump and pg_restore but it seems there is no solution for this problem.
How can I restore the database?
I just mentioned the above settings in this question but still if more code is required then tell me I'll update my question with that information. Thank you
Here is a message after hitting ALTER ROLE app_admin SET lc_messages = 'C'; command and restore.
postgres=# ALTER ROLE app_admin SET lc_messages = 'C';
ALTER ROLE
postgres=# \q
C:\Users\enosh>pg_restore -v -U app_admin -d pre_rulings -t test_eu3 test_eu3.sql
pg_restore: connecting to database for restore
Password:
pg_restore: implied data-only restore
pg_restore: processing data for table "public.test_eu3"
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 2915; 0 3097390 TABLE DATA test_eu3 postgres
pg_restore: error: could not execute query: ERROR: relation "public.test_eu3" does not exist
Command was: COPY public.test_eu3 (id, "national", item_day, item_hs_all, item_hs2, item_hs4, item_hs6, item_place, image_amount, img_name_all, item_image, other_info, org_discription, org_discription2, eng_discription, eng_discription2, id2) FROM stdin;
pg_restore: warning: errors ignored on restore: 1
The parameter lc_messages is set to a language that your terminal cannot display. There are two options:
If you connect to PostgreSQL with a superuser, you can use the PGOPTIONS environment variable to override the setting during your pg_restore. On Unix-like systems, that would work as follows:
export PGOPTIONS='-c lc_messages=C'
pg_restore ...
On Windows, you can use
set PGOPTIONS="-c lc_messages=C"
pg_restore ...
This will produce English error messages.
If you don't connect to PostgreSQL with a superuser, you are not allowed to change lc_messages. In that case, your only option is to ask the database administrator to change lc_messages in postgresql.conf (and reload PostgreSQL), or to set the parameter on your user:
ALTER ROLE app_admin SET lc_messages = 'C';
A database's backup file created in Windows 7 with:
pg_dump -U postgres -Fc [db_name] >D:\[db_backup_file].sql
Then I dropped it and restored it to test the process with:
pg_restore -U postgres -C -d postgres D:\[db_backup_file].sql
Everything worked fine.
However as I tried to restore it in Ubuntu 20.04 in a different device, I got an error:
could not execute query: ERROR: invalid locale name: (same as here)
So I followed the given instructions creating the database,
sudo -u postgres psql
create database [db_name];
and then I placed in the terminal the following command to restore backup:
pg_restore -U postgres -d postgres /home/../../[db_backup_file].sql
But again I got errors, as many were the tables, multiplied by four.
So for every table I get the following errors:
pg_restore: from TOC entry 315; 1259 29971 TABLE [table_name] postgres
pg_restore: error: could not execute query: ERROR: relation [table_name] already exists
Command was: CREATE TABLE public.[table_name] (
[pkey_column_name] integer NOT NULL,
.......
.......
.......
.......
.......
.......
);
pg_restore: from TOC entry 314; 1259 29969 SEQUENCE [table_name]_[pkey_column_name]_seq postgres
pg_restore: error: could not execute query: ERROR: relation "[table_name]_[pkey_column_name]_seq" already
exists
Command was: CREATE SEQUENCE public.[table_name]_[pkey_column_name]_seq
AS integer
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;
pg_restore: from TOC entry 3522; 0 29971 TABLE DATA [table_name] postgres
pg_restore: error: COPY failed for table "[table_name]": ERROR: duplicate key value violates unique constraint "[table_name]_pkey"
DETAIL: Key ([pkey_column_name])=(1) already exists.
CONTEXT: COPY [table_name], line 1
pg_restore: from TOC entry 3267; 2606 29976 CONSTRAINT [table_name] [table_name]_pkey postgres
pg_restore: error:
could not execute query: ERROR: multiple primary keys for table "[table_name]" are not allowed
Command was: ALTER TABLE ONLY public.[table_name]
ADD CONSTRAINT [table_name]_pkey PRIMARY KEY ([pkey_column_name]);
When the tables were created the primary keys - if something has to do with it - were defined as auto increment, in the form of:
CREATE TABLE [table_name] (
[pkey_column_name] serial primary key,
.......
.......
.......
.......
.......
.......
);
Can anyone please help me about that?
EDIT: Actually the missing codepage type in the first error in my post yesterday was "Greek_Greece.1253". I used the locale -a command as you said, and I saw that among others my Ubuntu OS has en_US.UTF-8 and el_GR.UTF-8. So I'm wondering if the issue could be that incompatibility between Windows and Ubuntu character sets. If yes, how you thing that I could manage it?
Luckily the windows 7 device from which the backup files come from is still in use, and so the databases are active. But what I tried to create again the databases giving for LC_COLLATE and LC_CTYPE values compatible with ubuntu, didn't work.
EDIT 2: Finally it was that windows-linux incompatibility in characters encoding.
When I tried to use en_US.UTF-8 or el_GR.UTF-8 with encoding parameter as following:
pg_dump -E en_US.UTF-8 -U postgres -Fc [db_name] > D:\[backup_file].sql
I was getting:
pg_dump: invalid client encoding "en_US.UTF-8" specified
Then I tried to create in ubuntu the db before I restore it, under the command:
CREATE DATABASE database_name WITH ENCODING 'utf8' LC_COLLATE='el_GR.utf8' LC_CTYPE='el_GR.utf8' TEMPLATE template0;
and then:
pg_restore -U postgres -d postgres ~/../../backup_file.sql
but I got the same batch of errors I had in the initial post.
So the solution was to create a new database in windows, but now under 'C' char encoding (POSIX wouldn't be accepted), copy the tables from one database to another:
pg_dump -U postgres -t [table_name] [database_name] | psql -U postgres -d [database_name]
and then dump the newly created db, and restore it in ubuntu environment.
It could be that your Ubuntu OS does not have an en_US.UTF-8 locale. You can check for this by using this command in terminal:
locale -a # list all locales known to OS
If you cannot find the locale in the list, try making a new one according to this post
EDIT
With the additional information that the Windows encoding is Greek_Greece.1253, it still sounds like there is a mismatch. According to the pg_dump docs, you can explicitly set the encoding using the -E option. Probably you want to set it to something that Ubuntu can handle (i.e. en_US.UTF-8 or el_GR.UTF-8)
-E encoding
--encoding=encoding
Create the dump in the specified character set encoding. By default, the dump is
created in the database encoding. (Another way to get the same result is to set the
PGCLIENTENCODING environment variable to the desired dump encoding.)
I every now and then restores a db-dump. To save time and I want to exclude some large un-needed tables.
I try to use the -L (use-list) option but when running the pg_restor command it does not work.
I get the same result even if I'm using a un-edited list.
This is the command I use:
pg_restore -v -U [user] -L "restore.pgdump.list" -d [dbname]-h localhost -p 5432 [dump-file]
This is the result:
pg_restore: [archiver] WARNING: line ignored: ÿþ
pg_restore: connecting to database for restore
Password:
pg_restore: implied data-only restore
If I go with the same command without the L-option the database is restored correctly.
Would much appreciate your help
Peter
My goal is to dump the table structure for all tables in the database without any contents.
I have some Redshift tables that include reserved keywords. If I try to dump them I get this error:
pg_dump -sc --quote-all-identifiers -h example.com -p 5439 -d redacted -U redacted --table public.mytable > ./blah.txt
pg_dump: [archiver (db)] query failed: ERROR: syntax error at or near "stdout"
LINE 1: ... "status", "date") TO stdout;
^
pg_dump: [archiver (db)] query was: COPY "public"."mytable" ("status", "date") TO stdout;
If I try to export just the table structure using pg_dump -sc --quote-all-identifiers -h example.com -p 5439 -d redacted -U redacted --table public.mytable > ./blah.txt then the command succeeds, but the resulting file contains this create table statement:
CREATE TABLE "mytable" (
);
With nothing inside the parentheses aside from a new line.
The pg_dump version is pg_dump (PostgreSQL) 9.5.8 on Ubuntu 16.04.
The issue affects several different tables and one of the things in common among those tables is having columns named with words on the psql reserved words list. Maybe there's another explanation for why this is happening, so I'd love to hear alternate theories of the problem.
While it would be nice, I cannot change the names of the columns in the tables.
Strongly recommend that you shouldn't use pg_dump for Redshift. Try the UNLOAD command instead.
https://docs.aws.amazon.com/redshift/latest/dg/r_UNLOAD.html
I am having a problem upgrading my Postgresql 9.2 database to 9.4. The problem I have is that I have these tables that are incompatible for an upgrade:
public.pg_ts_dict.dict_init
public.pg_ts_dict.dict_lexize
public.pg_ts_parser.prs_start
public.pg_ts_parser.prs_nexttoken
public.pg_ts_parser.prs_end
public.pg_ts_parser.prs_headline
public.pg_ts_parser.prs_lextype
Postgresql says that I should delete these tables and the upgrade should work. I am currently trying to figure out how to do this.
Postgresql has pg_catalog.pg_ts_parser and pg_catalog.pg_ts_dict. These are system catalogs and can absolutely not be removed, nor do I want to. I want to remove the public.pg_ts_* tables.
More specifically I want to dump the tables, upgrade the database, and then restore both public.pg_ts_parser and public.pg_ts_dict. However every time I try to dump or drop the tables, it defaults to the system catalog. Which I don't want. How can I specify these exact tables? Thanks for any help in advance.
-------EDIT------
Here are the commands I am running to dump the tables.
pg_dump -Fc -t public.pg_ts_dict -t public.pg_ts_parser > file.dump
pg_dump: No matching tables were found
Here is a variation
pg_dump -Fc -t pg_ts_dict -t pg_ts_parser > file.dump
The second variation contains the dump of the system catalog pg_ts_dict and parser not the public version. However, it is very confusing because the contents of the file.dump contains these lines of code among # signs and ^ signs.
DROP TABLE pg_catalog.pg_ts_dict;
^#^#^#pg_catalog^#^#^#^#^#^#^H^#^#^#postgres^#^D^#^#^#true^A^A^#^#^#^C^#^#^#^#^#^#^#^#^# ^G^#^#^#^#^#^#^#^#^A^#^#^#0^#^A^#^#^#0^#
^#^#^#pg_ts_dict^#^C^#^#^#ACL^#^A^#^#^#^#<86>^#^#^#REVOKE ALL ON TABLE pg_ts_dict FROM PUBLIC;
REVOKE ALL ON TABLE pg_ts_dict FROM postgres;
GRANT SELECT ON TABLE pg_ts_dict TO PUBLIC;
^#^#^#^#^#^A^A^#^#^#^#
^#^#^#pg_catalog^A^A^#^#^#^#^H^#^#^#postgres^#^E^#^#^#false^#^B^#^#^#54^A^A^#^#^#^C^#^#^#^#^#^#^#^#^#7^#^#^#^#^#^#^#^#^#^D^#^#^#1259^#^D^#^#^#3601^#^L^#^#^#pg_ts_parser^#^E^#^#^#TABLE^#^B^#^#^#^#ö^#^#^#CREATE TABLE pg_ts_parser (
prsname name NOT NULL,
prsnamespace oid NOT NULL,
prsstart regproc NOT NULL,
prstoken regproc NOT NULL,
prsend regproc NOT NULL,
prsheadline regproc NOT NULL,
prslextype regproc NOT NULL
Not sure what to make of this.
Your call should actually work as is:
pg_dump -Fc -t public.pg_ts_dict -t public.pg_ts_parser > file.dump
You can use a wildcard to include all tables starting with pg_ts_.
pg_dump -Fc -t 'public.pg_ts_*' > file.dump
On the Linux shell, you may need the extra quotes. (Related question on dba.SE.) Remove the quotes in Windows.
To make it abundantly clear you could exclude the same tables from pg_catalog explicitly. Normally, this is not necessary, but something seems to be abnormal in your case.
pg_dump -Fc -t 'public.pg_ts_*' -T 'pg_catalog.pg_ts_*' > file.dump
The documentation:
Also, you must write something like -t sch.tab to select a table in a
particular schema, rather than the old locution of -n sch -t tab.