I am dumping a large Postgres table like this:
pg_dump -h myserver -U mt_user --table=mytable -Fc -Z 9 --file mytable.dump mydb
The above creates a mytable.dump file. I now want to restore this dump into a new table called mytable_restored.
How can I use the pg_restore command to do this?
There is no pg_restore option to rename tables.
I would do it like this:
-- create a table that is defined like the original
CREATE TABLE mytable_restored (LIKE mytable INCLUDING ALL);
-- copy the table contents
COPY mytable TO '/tmp/mytable.dmp';
COPY mytable_restored FROM '/tmp/mytable.dmp';
you can append sed to your pg_dump so that while pg_dump outputs the table name, sed will replace it with the right one.
pg_dump mytable mydb | sed 's/mytable/mytable_restored/g' > mytable_restored.dump
My answer may not the exact answer for the question but for my case (directory dump with many tables), I have improved Laurenz's answer, it may be useful.
In my case I have just needed two tables to restore to another location.
First I created the tables in new location.
I find the table data in the toc.dat file.
pg_restore -Fd <dir_name> -l | grep <table_name>
This will probably return more than single line. Then we look for "TABLE DATA" line. After getting [file_number].dat.gz file and use it in copy command like below in psql.
COPY <table_name> from PROGRAM 'zcat /<path>/<file_number>.dat.gz'
Related
What I want:
I want a pg_dump of a database (let's call the database as 'test').
In this pg_dump I want only the tables without the following: data, triggers, functions, sequences, etc.
What I am doing to get what I want:
The command I run is as follows:
pg_dump -U postgres -s test > dump_test.sql
What I am observing:
Then when I try to restore this dump on another server as follows:
pg_dump -U postgres new_amazing_test < dump_test.sql
I notice that part of the output of running the above command says the following:
CREATE TRIGGER
CREATE FUNCTION
CREATE SEQUENCE
CREATE INDEX
What I actually want:
All I want is the table itself and not these triggers, functions, sequence and indexes. How do I only get the tables only?
Other things I have tried/considered:
I have tried doing this:
pg_dump -U postgres -s -schema=\dtmvE test > dump_test.sql
but it didn't work because the pattern needs to be a name not a \d pattern.
See here: https://www.postgresql.org/docs/13/app-pgdump.html for information on -n pattern option.
One thing that may solve it is to use multiple switches like this:
pgdump -t mytable1 -t mytable2 -t mytable3 ... -t mytableN > dump_test.sql
However, the above solution is impractical because I have some 70+ tables on my database.
Other relevant info:
PostgreSQL version is 13.1
Ubuntu version v16.04 (I have also tried this on Ubuntu v18.04)
I would dump everything with a custom format schema-only dump (-F c -s) and run pg_restore -l on the resulting dump. That gives you a table of contents. Delete everything except the tables from that file and use it as input to pg_restore -L to restore exactly those items from the archive that you need.
This may not be as simple as you have hoped for, but it is certainly simpler than writing tons of -t options, and you may be able to automatize it.
you can use the flag --section as described in the postgres documentation
--section=sectionname
Only dump the named section. The section name can be pre-data, data, or post-data. This option can be specified more than once to select multiple sections. The default is to dump all sections.
The data section contains actual table data, large-object contents, and sequence values. Post-data items include definitions of indexes, triggers, rules, and constraints other than validated check constraints. Pre-data items include all other data definition items.
example:
pg_dump --schema-only --section=pre-data
I have a database with several tables and I want to add the data and structure of a selection of them to a different database containing already different tables.
I have created the dump file in the following way:
"C:\Program Files\PostgreSQL\9.1\bin\pg_dump.exe" -U postgres -w DBName -t table1 -t Table2 -t Table3 > "E:\Dump.sql"
This works fine and I have E:\Dump.sql with the dump of the three tables in it.
I have tried to make a restore with the following code:
"C:\Program Files\PostgreSQL\9.6\bin\pg_dump.exe" -C -U User -t table1 -t Table2 -t Table4 destdb < Dump.sql
but I get the error
no matching tables were found
.
Where am I doing wrong?
https://www.postgresql.org/docs/current/static/backup-dump.html
pg_dump is meant for **taking* backups, next section of manual is for restoring a backup:
Text files created by pg_dump are intended to be read in by the psql
program. The general command form to restore a dump is
psql dbname < infile
Also you mention three tables, two of which are mixed case - in order for it to work, you have to double quote it I believe. But anyway - restoring for you file would be:
psql YOURDBNAME -f E:\Dump.sql
I have a Postgres RDS instance for one of my apps.
I need to copy 3 tables from it to a similar clone of the database.
I see mytable_id_seq tables also, which now I know are called sequences in postgres terminology.
When I created a dump of those three tables, and restore them, do I have to do anything with the _id_seq sequences ?
Do I have to restore them too, for the dump data to work as it did in the original table?
When you restore the entire database from a dump, it contains CREATE SEQUENCE statements by default. These statements initialize sequences to the proper state. But if you make a partial dump, with only selected tables, you must set sequences manualy.
Assuming, that your table's name is "clip", you can check the current value using this query:
SELECT last_value FROM clip_id_seq
And if you want to update the sequence after restore, you can do it with this simple query:
SELECT SETVAL('clip_id_seq', SELECT MAX(id) FROM clip)
pg_dump -d database_name -t mg_cnd -F c > database_backup.sql
pg_restore -U database_user --data-only -d database_name -t mg_cnd -F c <file_location>
-d = database name
-t = table name
-F = Format
c = plain
-U = User
--data-only = transfer only table data
pg_dump documentation
pg_restore documentation
I have taken a dump of a database named temp1, by using the follwing command
$ pg_dump -i -h localhost -U postgres -F c -b -v -f pub.backup temp1
Now I want to restore the dump in a different database called "db_temp" , but in that I just want that all the tables should be created in a "temp_schema" ( not the default schema which is in the fms temp1 database ) which is in the "db_temp" database.
Is there any way to do this using pg_restore command?
Any other method also be appreciated!
A quick and dirty way:
1) rename default schema:
alter schema public rename to public_save;
2) create new schema as default schema:
create schema public;
3) restore data
pg_restore -f pub.backup db_temp [and whatever other options]
4) rename schemas according to need:
alter schema public rename to temp_schema;
alter schema public_save rename to public;
There is a simple solution:
Create your backup dump in plain SQL format (format "p" using the parameter --format=p or -F p)
Edit your pub.backup.sql dump with your favorite editor and add the following two lines at the top of your file:
create schema myschema;
SET search_path TO myschema;
Now you can restore your backup dump with the command
psql -f pub.backup.sql
The set search_path to <schema> command will set myschema as the default, so that new tables and other objects are created in this schema, independently of the "default" schema where they lived before.
There's no way in pg_restore itself. What you can do is use pg_restore to generate SQL output, and then send this through for example a sed script to change it. You need to be careful about how you write that sed script though, so it doesn't match and change things inside your data.
Probably the easiest method would be to simply rename the schema after restore, ie with the following SQL:
ALTER SCHEMA my_schema RENAME TO temp_schema
I believe that because you're using the compressed archive format for the output of pg_dump you can't alter it before restoring. The option would be to use the default output and do a search and replace on the schema name, but that would be risky and could perhaps cause data to be corrupted if you were not careful.
If you only have a few tables then you can restore one table at a time, pg_restore accepts -d database when you specify -t tablename. Of course, you'll have to set up the schema before restoring the tables and then sort out the indexes and constraints when you're done restoring the tables.
Alternatively, set up another server on a different port, restore using the new PostgreSQL server, rename the schema, dump it, and restore into your original database. This is a bit of a kludge of course but it will get the job done.
If you're adventurous you might be able to change the database name in the dump file using a hex editor. I think it is only mentioned in one place in the dump and as long as the new and old database names are the same it should work. YMMV, don't do anything like this in a production environment, don't blame me if this blows up and levels your home town, and all the rest of the usual disclaimers.
Rename the schema in a temporary database.
Export the schema:
pg_dump --schema-only --schema=prod > prod.sql
Create a new database. Restore the export:
psql -f prod.sql
ALTER SCHEMA prod RENAME TO somethingelse;
pg_dump --schema-only --schema=somethingelse > somethingelse.sql
(delete the database)
For the data you can just modify the set search_path at the top.
As noted, there's no direct support in pg_dump, psql or pg_restore to change the schema name during a dump/restore process. But it's fairly straightforward to export using "plain" format then modify the .sql file. This Bash script does the basics:
rename_schema () {
# Change search path so by default everything will go into the specified schema
perl -pi -e "s/SET search_path = $2, pg_catalog/SET search_path = $3, pg_catalog, $2;/" "$1"
# Change 'ALTER FUNCTION foo.' to 'ALTER FUNCTION bar.'
perl -pi -e 's/^([A-Z]+ [A-Z]+) '$2'\./$1 '$3'./' "$1"
# Change the final GRANT ALL ON SCHEMA foo TO PUBLIC
perl -pi -e 's/SCHEMA '$2'/SCHEMA '$3'/' "$1"
}
Usage:
pg_dump --format plain --schema=foo --file dump.sql MYDB
rename_schema dump.sql foo bar
psql -d MYDB -c 'CREATE SCHEMA bar;'
psql -d MYDB -f dumpsql
The question is pretty old, but maybe can help some one.
Streaming the output of pg_restore to sed and replace the schema name in order to import the dump to a different schema.
Something like:
pg_restore ${dumpfile} | \
sed -e "s/OWNER TO ${source_owner}/OWNER TO ${target_owner}/" \
-e "s/${source_schema}/${target_schema}/" | \
psql -h ${pgserver} -d ${dbname} -U ${pguser}
How can I generate the DDL of a table programmatically on Postgresql? Is there a system query or command to do it? Googling the issue returned no pointers.
Use pg_dump with this options:
pg_dump -U user_name -h host database -s -t table_or_view_names -f table_or_view_names.sql
Description:
-s or --schema-only : Dump only ddl / the object definitions (schema), without data.
-t or --table Dump : Dump only tables (or views or sequences) matching table
Examples:
-- dump each ddl table elon build.
$ pg_dump -U elon -h localhost -s -t spacex -t tesla -t solarcity -t boring > companies.sql
Sorry if out of topic. Just wanna help who googling "psql dump ddl" and got this thread.
You can use the pg_dump command to dump the contents of the database (both schema and data). The --schema-only switch will dump only the DDL for your table(s).
Why would shelling out to psql not count as "programmatically?" It'll dump the entire schema very nicely.
Anyhow, you can get data types (and much more) from the information_schema (8.4 docs referenced here, but this is not a new feature):
=# select column_name, data_type from information_schema.columns
-# where table_name = 'config';
column_name | data_type
--------------------+-----------
id | integer
default_printer_id | integer
master_host_enable | boolean
(3 rows)
The answer is to check the source code for pg_dump and follow the switches it uses to generate the DDL. Somewhere inside the code there's a number of queries used to retrieve the metadata used to generate the DDL.
Here is a good article on how to get the meta information from information schema,
http://www.alberton.info/postgresql_meta_info.html.
I saved 4 functions to mock up pg_dump -s behaviour partially. Based on \d+ metacommand. The usage would be smth alike:
\pset format unaligned
select get_ddl_t(schemaname,tablename) as "--" from pg_tables where tableowner <> 'postgres';
Of course you have to create functions prior.
Working sample here at rextester