psql has a -q / --quiet option (environment variable QUIET). pg_restore does not have a quiet option. Is there any way to make pg_restore not verbosely show the SQL commands that it's executing?
# e.g., here's the verbose output that I don't want to see:
$ pg_restore --cluster 8.4/mycluster mycluster.dump
---- PostgreSQL database dump
--
SET statement_timeout = 0;SET client_encoding = 'UTF8';
SET standard_conforming_strings = off;SET check_function_bodies = false;
...
--
-- Name: data_src; Type: TABLE; Schema: public; Owner: postgres; Tablespace:--
CREATE TABLE data_src (
...
The question seems to imply that pg_restore is executing these SQL commands and you wouldn't want to see them in the output. But outputting them is what it's only supposed to do.
pg_restore has two modes of operation, with or without connecting to a database. When it's called without a database (-d option) as shown in the question:
$ pg_restore --cluster 8.4/mycluster mycluster.dump
then its sole purpose is to output a set of SQL commands in plain text that should be fed to an SQL interpreter to restore the database. Those SQL commands form a coherent set without any concept of verbosity, and they are not executed by pg_restore itself. They're generally redirected into a file for later execution or piped into psql for immediate execution.
You can redirect stdout to a file:
pg_restore --cluster 8.4/mycluster mycluster.dump > pg_restore.log
Or provide the -d option, but what you want is either -f or -d
pg_restore -f pg_restore.sql --cluster 8.4/mycluster mycluster.dump
pg_restore -d yourdatabase --cluster 8.4/mycluster mycluster.dump
Related
I created a serverless Aurora-Postgresql RDS cluster on AWS in a VPC, and a Linux EC2 instance to access it from, also in the VPC. I have a binary file dump.bin I dumped from a prior database with pg_dump -Fc that I want to use to restore the new database to the state of the old one. Inside the EC2 instance, I can run pg_dump against the new database with no issues, which creates a dump of the unconfigured database. I can even run pg_restore --version, and it prints pg_restore (PostgreSQL) 10.17. But when I run pg_restore with arguments to connect to the database and actually try to commit the restore, the program simply hangs indefinitely with no output.
This is the command I am running:
pg_restore --host=<rds-instance-dns-host>.us-east-2.rds.amazonaws.com --port=5432 --username=postgres --file=dump.bin --verbose
I think you mistook --file option for FILE argument in pg_restore:
$ pg_restore --help
pg_restore restores a PostgreSQL database from an archive created by pg_dump.
Usage:
pg_restore [OPTION]... [FILE]
…
-f, --file=FILENAME output file name (- for stdout)
…
Similarily from man pg_restore:
-f filename
--file=filename
Specify output file for generated script,
or for the listing when used with -l.
Use - for stdout.
The --file tells where to write output, an SQL script that would restore the database if run on it. It is not about from which file to restore. So your pg_restore does not know from which file to read, so it tries to read from its standard input.
So your command should rather look like this:
pg_restore --host=<rds-instance-dns-host>.us-east-2.rds.amazonaws.com \
--port=5432 --username=postgres --verbose dump.bin
Or, as I'd recommend, use parallel restore and output to a file, so you can examine it later in case of any errors:
pg_restore --host=<rds-instance-dns-host>.us-east-2.rds.amazonaws.com \
--jobs=8 --port=5432 --username=postgres --verbose dump.bin \
> /tmp/pg_restore.log 2>&1
I am trying to execute the following command to restore db in a Docker container:
cmd /c "docker exec -i database-container pg_restore -C -U postgres -d employee -v < C:\\backup\\employee.tar"
But it throws: "pg_restore: error: connection to database "employee" failed: FATAL: database "employee" does not exist" (If I create an empty database with the same name, it is created without any problem).
The reason may be:
When this option is used, the database named with -d is used only to issue the initial DROP DATABASE and CREATE DATABASE commands. All data is restored into the database name that appears in the archive.
So, is it possible to restore db using the same pg_restore command? Or should I modify it by adding drop and create commands to fix the problem?
Finally I concluded to drop and create db before restore. If you have a better solution you are welcome and I will give a try.
docker exec database-container bash -c "dropdb -U postgres employee"
docker exec database-container bash -c "createdb -U postgres employee"
cmd /c "docker exec -i database-container pg_restore -C -U postgres -d employee -v
< C:\\backup\\employee.tar"
You can let pg_restore drop and recreate the database before restoring your backup by providing both parameters --create --clean together.
--create
Create the database before restoring into it. If --clean is also specified, drop and recreate the target database before connecting to it.
See https://www.postgresql.org/docs/12/app-pgrestore.html
I'm running a PostgreSQL DB and want to compare the schema defined in schema.sql with the schema that is created on the DB on a regular basis. I.e. to verify that no manual changes on the remote are applied or that schema.sql actually reflects what's running on the DB.
What I'm currently doing is roughly:
create a dump from schema.sql using pg_virtualenv:
$ pg_virtualenv sh -c 'psql -U $PGDATABASE -f schema.sql
$ pg_dump --schema-only --no-comments --no-owner --no-privileges --schema=public -U $PGDATABASE -f schema_local.sql'
create a dump from the remote
$ pg_dump --schema-only --no-comments --no-owner --no-privileges --schema=public -U $PGREMOTE -f schema_upstream.sql
strip out comments, SET, emtpy lines, etc. and diff them
$ diff -u <(grep -Ev '^--|^SET|^$$' schema_local.sql) <(grep -Ev '^--|^SET|^$$' schema_upstream.sql)
That works reasonably well and I can easily spot schema changes in the diff if they occur. But the solution seems a bit hacky and I wonder if there's a better way to achieve the same thing?
When I use pg_dump to export schema from a database, it adds the following line at the beginning:
SELECT pg_catalog.set_config('search_path', '', false);
Is it possible set an option where pg_dump will not add this line? It is causing issues later when I try to execute other SQL commands, without the schema qualifier.
This is the pg_dump command I am using right now:
pg_dump -O -x -h <db-host> -p <db-port> -U <db-user> -d <db-name> --schema public --schema-only > public-schema.sql
No, there is no such option.
I recommend that you restore a dump with psql -f dumpfile rather than using \i to execute it in the current session.
I am trying to setup a script to take a copy of a database from one server to another.
Thanks to this post Copying PostgreSQL database to another server I have found a way to do that.
But what I need to do is change the name of the database during the copy.
I have thought about using sed and doing a simple text replace. But I am worried that this could corrupt the database.
Does any one know the proper way of doing this?
As requested here are the commands I am using
pg_dump -C -U remoteuser -h remoteServer dbname | psql -h localhost -U localadmin template1
Just restore to a different database. For pg_restore of -Fc dumps from pg_dump's custom format:
createdb newdbname
pg_restore --dbname newdbname database.dump
For SQL-format dumps not created with the -C option to pg_dump:
createdb newdbname
psql -f database_dump.sql newdbname
If you're streaming the dump from a remote host, just omit -f database_dump.sql as the dump data is coming from stdin.
You can't easily CREATE DATABASE in the same command as your restore, because you need to connect to a different DB like template1 in order to create the new DB. So in your example you might:
psql -h localhost -U localadmin template1 -c 'CREATE DATABASE newdb;'
pg_dump -U remoteuser -h remoteServer dbname | psql -h localhost -U localadmin newdb
Note the omission of the -C flag to pg_dump.
The first command is just the longhand way of writing createdb -h localhost -U localadmin newdb.
Update: If you're stuck with a pg_dump created with the -C flag you can indeed just sed the dump so long as you're extremely careful. There should only be four lines (one a comment) at the start of the file that refer to the database name. For the database name "regress" dumped with Pg 9.1's pg_dump -C:
--
-- Name: regress; Type: DATABASE; Schema: -; Owner: craig
--
CREATE DATABASE regress WITH TEMPLATE = template0 ENCODING = 'UTF8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8';
ALTER DATABASE regress OWNER TO craig;
\connect regress
This can be transformed quite safely with three (or four if you want to rewrite the comment) very specific sed commands. Do not just do a global find and replace on the database name, though.
sed \
-e 's/^CREATE DATABASE regress/CREATE DATABASE newdbname/' \
-e 's/^ALTER DATABASE regress/ALTER DATABASE newdbname/' \
-e 's/^\\connect regress/\\connect newdbname/' \
-e 's/^--Name: regress/--Name: newdbname/'
This should be a last resort; it's much better to just dump without -C.