After dumping a database using pg_dump like so:
pg_dump --verbose --host=<host> --username=<user> -W --encoding=UTF-8 -j 10 --file=dump_bak --format=d --dbname=<database>
and trying to reimport it with:
pg_restore -d <database> --host=<host> -n public --username=<user> -W --exit-on-error --format=d -j 10 --verbose dump_bak
…we are missing some of our Primary Keys. It looks like a few have been restored, but not all.
Any ideas?
It seems that Postgres is a bit buggy there concerning existing connections. We have found a workaround:
Dump it with:
pg_dump --verbose --host=<dbhost> --username=<username> --encoding=UTF-8 --file=<dumpfile> --format=d --jobs=10 --dbname=<dbname>
Restore it with:
export PGPASSWORD="<pwd>"
#prevent new connections, kill existing connections
sudo -u postgres /usr/bin/psql -q -c "update pg_database set datallowconn = 'false' where datname = '<dbname>'; SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '<dbname>';"
#kick db
sudo -u postgres dropdb <dbname>
#recreate it
sudo -u postgres createdb <dbname>
#allow connections again
sudo -u postgres /usr/bin/psql -q -c "update pg_database set datallowconn = 'true' where datname = '<dbname>';"
#import data from dump
pg_restore --verbose -d <dbname> --host=$1 --username=<username> -j 4 <dumplocation>
Related
I want to execute these lines in the below .sh file:
#!/bin/sh +x
sudo su postgres
psql -U postgres -tc "SELECT 1 FROM pg_database WHERE datname = 'test_db'" | grep -q 1 || psql -U postgres -c "CREATE DATABASE test_db"
But only 'sudo su postgres' is executing and the 2nd line is not executing. Can someone help me execute those 2 lines
Just pass your psql command to su:
sudo su postgres -c "psql -U postgres -tc \"SELECT 1 FROM pg_database WHERE datname = 'test_db'\" | \
grep -q 1 || psql -U postgres -c \"CREATE DATABASE test_db\""
If you want to pass the datname value as a parameter, replace it by $1 in the script ($1, $2, etc. expand as the first, second, etc. parameters you pass to your script):
sudo su postgres -c "psql -U postgres -tc \"SELECT 1 FROM pg_database WHERE datname = '$1'\" | \
grep -q 1 || psql -U postgres -c \"CREATE DATABASE $1\""
and call your script like this:
./script.sh test_db
You can even have optional parameters. For instance, if you want an optional parameter for the table name (pg_database in your example):
db=pg_database
if [ -n "$2" ]; then db="$2"; fi
sudo su postgres -c "psql -U postgres -tc \"SELECT 1 FROM $db WHERE datname = '$1'\" | \
grep -q 1 || psql -U postgres -c \"CREATE DATABASE $1\""
and call your script like this:
./script.sh test_db
to use the default, else:
./script.sh test_db other_pg_database
I suggest you update your sudo configuration so you can directly express that you only need to as postgres and not root:
sudo -u postgres ...
For example:
user host = (postgres) command
where user is your user, host is the host name you want this apply (ALL?), and command is the name of your command (ALL?) possible prefixed with "NOPASSWD: " if you don't want to require a password.
Then do the action and deal with the error if needed instead of guarding against it:
sudo -u postgres bash -c "psql -U postgres -c "CREATE DATABASE $1"
Possible single quoting the $1 and if you want that to be robust, escape any single quotes in the database name. I showed you a shell (bash) in the above, so can easily tag on error handling.
I take backup using
pg_dump db_production > postgres_db.dump
and then I copy it to localhost using scp.
Now when I import on my local db it gives an error
pg_restore: [archiver] input file appears to be a text format dump. Please use psql.
by using commad line
pg_restore -d db_development postgres_db.dump
From the pg_dump documentation:
Examples
To dump a database called mydb into a SQL-script file:
$ pg_dump mydb > db.sql
To reload such a script into a (freshly created) database named newdb:
$ psql -d newdb -f db.sql
To dump a database into a custom-format archive file:
$ pg_dump -Fc mydb > db.dump
To dump a database into a directory-format archive:
$ pg_dump -Fd mydb -f dumpdir
To reload an archive file into a (freshly created) database named newdb:
$ pg_restore -d newdb db.dump
From the pg_restore documentation:
Examples
Assume we have dumped a database called mydb into a custom-format dump file:
$ pg_dump -Fc mydb > db.dump
To drop the database and recreate it from the dump:
$ dropdb mydb
$ pg_restore -C -d postgres db.dump
The answer above didn't work for me, this worked:
psql db_development < postgres_db.dump
In order to create a backup using pg_dump that is compatible with pg_restore you must use the --format=custom / -Fc when creating your dump.
From the docs:
Output a custom-format archive suitable for input into pg_restore.
So your pg_dump command might look like:
pg_dump --file /tmp/db.dump --format=custom --host localhost --dbname my-source-database --username my-username --password
And your pg_restore command:
pg_restore --verbose --clean --no-acl --no-owner --host localhost --dbname my-destination-database /tmp/db.dump
For me when i try to restore from remote host i used
psql -U username -p 5432 -h 10.10.10.1 -d database < db.dump
worked fine. And if not remote just following command worked.
psql -d database < db.dump
For me, It's working like this one.
C:\Program Files\PostgreSQL\12\bin> psql -U postgres -p 5432 -d dummy -f C:\Users\Downloads\d2cm_test.sql
If you restore .SQL file.
Create a new database in pgAdmin.
Go to the terminal and navigate the folder/directory where your .sql file is located. And then write the following command in terminal.
Syntax:
supername user postgres psql newDatabasename < inputfile.sql
Examaple:
sudo -u postgres psql newDb < restoreDb.sql
I've got same error when tried to backup db with DBeaver. If anyone uses DBeaver interface instead of command line on Windows, make sure your selected format as tar during backup and restore settings.
If you have a full DB dump:
PGPASSWORD="your_pass" psql -h "your_host" -U "your_user" -d "your_database" -f backup.sql
If you have schemas kept separately, however, that won't work. Then you'll need to disable triggers for data insertion, akin to pg_restore --disable-triggers. You can then use this:
cat database_data_only.gzip | gunzip | PGPASSWORD="your_pass" psql -h "your_host" -U root "your_database" -c 'SET session_replication_role = replica;' -f /dev/stdin
On a side note, it is a very unfortunate downside of postgres, I think. The default way of creating a dump in pg_dump is incompatible with pg_restore. With some additional keys, however, it is. WTF?
if you use pg_dump with -Fp to backup in plain text format, use following command:
cat db.txt | psql dbname
to copy all data to your database with name dbname
psql -U <username> -d <database-name> -h <host-name> -f <backup.sql>
Providing a simple one line answer which worked for me and will work for you too for most cases
psql -U username -d database_name < dump_file.sql
If above gives role related errors then replace username with postgres.
psql -U postgres -d database_name < dump_file.sql
Probably when you create a backup you want to restore it in another network or create a remote restoration.
We need to create a backup file using the --format=custom [-Fc] to restore it using pg_restore. We can use a connection string postgresql://<user>:<pass>#localhost:5432/<dbname> and replace <user>, <pass>, and <dbname> with your information.
pg_dump -v -Fc \
postgresql://<user>:<pass>#localhost:5432/<dbname> \
> db-20211122-163508.sql
To restore we will call it using --clean [-c] and --create [-C] to drop the database before restoring. Replace <user>, <host>, <port>, and <dbname> with your information.
pg_restore -vcC \
-U <user> \
-h <host> \
-p <port> \
-d <dbname> \
< db-20211122-163508.sql
If you backup with this way, I think this will be more easy to import database.
pg_dump -h (remote db address) -a --column-inserts -U postgres (database name) > (file name).sql
For import,
psql
-f (file name).sql
--host (remote db address)
--port 5432
--username postgres
--password (your password)
--dbname (database you want to import)
I've been struggling with this as well. This is the combination of dump & restore commands that worked for me:
pg_dump -Ft -C -h database_host -U username database > DATA.dump
To restore
pg_restore -x --no-owner -d database DATA.dump
Remove the -x flag if you want to keep the same access privileges (ACLs) in your DB. You must have the same roles and users in the database for this.
https://www.postgresql.org/docs/15/app-pgdump.html
https://www.postgresql.org/docs/15/app-pgrestore.html
here is the solution,
pg_restore -U username -p 5432 -h 10.10.10.1 -d database_name < dump_file
I have tried multiple way to append log but not able to get.
for example.
pg_restore -U postgres -p5333 -d demodb < db_bkp_01_10_2019.dump >db_bkp_01_10_2019.log
pg_restore -U postgres -p5333 -d demodb < db_bkp_01_10_2019.dump 2>db_bkp_01_10_2019.log
pg_restore -U postgres -p5333 -d demodb < db_bkp_01_10_2019.dump 2&>db_bkp_01_10_2019.log
I want log related to restoration like what command executed on db while restoration.
You need to use the --verbose
Try this command
./pg_restore -U postgres -p 5333 -d demodb < db_bkp_01_10_2019.dump --verbose 2>db_bkp_01_10_2019.log
Took me sometime to figure this out, it works for me in psql 11 for log:
$pg_restore -h 127.0.0.1 -U user -p 5432 -d dbname -Fc import_data.sql>import.log 2>&1
I have using psql pipe command to copy a table from one database to another in Postgres. It is working fine. But I need to copy the table to a specific schema in the new database. I have gone through the documentation (used -n option for specifying schema name) but it is not working.
Command:
pg_dump -U postgres -h localhost -p 1212 -d dbname -t tablename -Ft | pg_restore -U postgres -h localhost -p 1213 -d dbname -n schemaname
you can't do it with pg_dump|pg_restore sequence. you need to alter table t set schema s; in restored db
I do it that this way:
pg_dump -U postgres -h localhost -p 1212 -d dbname -t tablename | sed "sed/oldschemaname/newschemaname/" | psql -U postgres -h localhost -p 1213 -d dbname -n schemaname
With the corresponding regular expression
Is there a way to create a backup of a single table within a database using postgres? And how? Does this also work with the pg_dump command?
Use --table to tell pg_dump what table it has to backup:
pg_dump --host localhost --port 5432 --username postgres --format plain --verbose --file "<abstract_file_path>" --table public.tablename dbname
If you are on Ubuntu,
Login to your postgres user sudo su postgres
pg_dump -d <database_name> -t <table_name> > file.sql
Make sure that you are executing the command where the postgres user have write permissions (Example: /tmp)
Edit
If you want to dump the .sql in another computer, you may need to consider skipping the owner information getting saved into the .sql file.
You can use pg_dump --no-owner -d <database_name> -t <table_name> > file.sql
pg_dump -h localhost -p 5432 -U postgres -d mydb -t my_table >
backup.sql
You can take the backup of a single table but I would suggest to take the backup of whole database and then restore whichever table you need. It is always good to have backup of whole database.
9 ways to use pg_dump
If you prefer a graphical user interface, you can use pgAdmin III (Linux/Windows/OS X). Simply right click on the table of your choice, then "backup". It will create a pg_dump command for you.
you can use this command
pg_dump --table=yourTable --data-only --column-inserts yourDataBase > file.sql
you should change yourTable, yourDataBase to your case
As an addition to Frank Heiken's answer, if you wish to use INSERT statements instead of copy from stdin, then you should specify the --inserts flag
pg_dump --host localhost --port 5432 --username postgres --format plain --verbose --file "<abstract_file_path>" --table public.tablename --inserts dbname
Notice that I left out the --ignore-version flag, because it is deprecated.
Use the following command to get the compressed version of the table dump :
pg_dump -h localhost -p 5432 -U <username> -d <dbname> -t <tablename> -Fc -f backup.out
Here is how I do it.
pg-dump -h localhost -U postgres -p 5432 -t table database > path/to/store/name.sql
and run it like this
psql -h localhost -U postgres -p 5432 database < path/to/store/name.sql