I can invoke psql like this:
psql postgres://...
How can I use pg_dump with a connection string in the format postgres://...?
Would be more convenient than breaking URI's into host, post, username, password.
Is there syntax for this?
pg_dump postgres://username:password#my_postgres_server:5432/databasename
I just downloaded a dump using this format.
Optionally you can add -f filename to specify a local filename.
This is for newer versions of pg_dump:
pg_dump -d postgres://username:password#my_postgres_server:5432/databasename > filename
Related
Using PostgreSQL v13.x, Windows 2019 Server
I'm using the following command either from a Powershell or command window
> pg_dump -h 127.0.0.1 -U postgres -W --format custom --file my_db_dump.sql my_db
(p/w prompted and entered)
The dump is created successfully. However it's full of extraneous non-printable control-sequence characters, e.g., ^A^N^#^X^#^#^#...., you name it. Post processing, e.g., dos2unix, :set ff=unix, :%!col -xb doesn't eliminate the characters. Is there a switch in pg_dump to control this? I didn't see it in the pg_dump documentation.
You have specified the custom format which is not plain SQL. It has structure and some compression. If you want plain SQL that is also available as a format.
So I regularly backup and restore databases and schema's using pgadmin4. I would like to do it with a batch file using commands as pg_dump and pg_restore. I however always fail to succeed in this and could use some help. The way I try to dump one schema (with data) is the following:
pg_dump -U postgres -F c -d database -n schema > mw2
Then I try to restore it with pg_restore:
pg_restore -U postgres -d otherdatabase -n schema mw2
First I tried to use .dump in stead of tar but the result stays the same; which is an empty schema in my database.
Or the following error message:
pg_restore: [archiver] input file does not appear to be a valid archive
https://www.postgresql.org/docs/current/static/app-pgrestore.html
--format=format Specify format of the archive. It is not necessary to specify the format, since pg_restore will determine the format
automatically. If specified, it can be one of the following:
custom, directory and tar
https://www.postgresql.org/docs/current/static/app-pgdump.html
--format=format
Selects the format of the output. format can be one of the following:
p plain Output a plain-text SQL script file (the default).
others are custom, directory and tar
in short - you used defualt plain format, which is meant for using with psql, not pg_restore. So either specify different format with pg_dump or use your file as
psql -f mw2.tar
When using pg_dump, you can use a postgres connection string to specify the host/database/username/password:
pg_dump postgres://someuser:somepassword#somehost.com:5432/somedatabase
I want to use the same sort of connection string for pg_restore:
pg_restore -f dump.dump postgres://userb:somepassword#somehost.com:5432/otherdatabase
But I get an error:
pg_restore: [archiver] could not open input file "postgres://userb:somepassword#somehost.com:5432/otherdatabase": No such file or directory
In PostgreSQL tools wherever you can specify a database name you can instead specify a connection string.
In the syntax for pg_restore the dbname is passed with a flag, not as a positional parameter:
$ pg_restore --help
pg_restore restores a PostgreSQL database from an archive created by pg_dump.
Usage:
pg_restore [OPTION]... [FILE]
General options:
-d, --dbname=NAME connect to database name
...
so you should be using:
pg_restore -d 'postgres://userb:somepassword#somehost.com:5432/otherdatabase' dump.dump
Yes, that user interface mismatch between pg_dump and pg_restore sucks, and I wish we could change it, but it's a bit late now.
I run scripts against my database like this...
psql -d myDataBase -a -f myInsertFile.sql
The only problem is I want to be able to specify in this command what schema to run the script against. I could call set search_path='my_schema_01' but the files are supposed to be portable. How can I do this?
You can create one file that contains the set schema ... statement and then include the actual file you want to run:
Create a file run_insert.sql:
set schema 'my_schema_01';
\i myInsertFile.sql
Then call this using:
psql -d myDataBase -a -f run_insert.sql
More universal way is to set search_path (should work in PostgreSQL 7.x and above):
SET search_path TO myschema;
Note that set schema myschema is an alias to above command that is not available in 8.x.
See also: http://www.postgresql.org/docs/9.3/static/ddl-schemas.html
Main Example
The example below will run myfile.sql on database mydatabase using schema myschema.
psql "dbname=mydatabase options=--search_path=myschema" -a -f myfile.sql
The way this works is the first argument to the psql command is the dbname argument. The docs mention a connection string can be provided.
If this parameter contains an = sign or starts with a valid URI prefix
(postgresql:// or postgres://), it is treated as a conninfo string
The dbname keyword specifies the database to connect to and the options keyword lets you specify command-line options to send to the server at connection startup. Those options are detailed in the server configuration chapter. The option we are using to select the schema is search_path.
Another Example
The example below will connect to host myhost on database mydatabase using schema myschema. The = special character must be url escaped with the escape sequence %3D.
psql postgres://myuser#myhost?options=--search_path%3Dmyschema
The PGOPTIONS environment variable may be used to achieve this in a flexible way.
In an Unix shell:
PGOPTIONS="--search_path=my_schema_01" psql -d myDataBase -a -f myInsertFile.sql
If there are several invocations in the script or sub-shells that need the same options, it's simpler to set PGOPTIONS only once and export it.
PGOPTIONS="--search_path=my_schema_01"
export PGOPTIONS
psql -d somebase
psql -d someotherbase
...
or invoke the top-level shell script with PGOPTIONS set from the outside
PGOPTIONS="--search_path=my_schema_01" ./my-upgrade-script.sh
In Windows CMD environment, set PGOPTIONS=value should work the same.
I'm using something like this and works very well:* :-)
(echo "set schema 'acme';" ; \
cat ~/git/soluvas-framework/schedule/src/main/resources/org/soluvas/schedule/tables_postgres.sql) \
| psql -Upostgres -hlocalhost quikdo_app_dev
Note: Linux/Mac/Bash only, though probably there's a way to do that in Windows/PowerShell too.
This works for me:
psql postgresql://myuser:password#myhost/my_db -f myInsertFile.sql
In my case, I wanted to add schema to a file dynamically so that whatever schema name user will provide from the cli, I will run sql file with that provided schema name.
For this, I replaced some text in the sql file. First I added {{schema}} in the file like this
CREATE OR REPLACE FUNCTION {{schema}}.usp_dailygaintablereportdata(
then replace {{schema}} dynamically with user provided schema name with the help of sed command
sed -i "s/{{schema}}/$pgSchemaName/" $filename
result=$(psql -U $user -h $host -p $port -d $dbName -f "$filename" 2>&1)
sed -i "s/$pgSchemaName/{{schema}}/" $filename
First replace is done, then target file is run and then again our replace is reverted back
I was facing similar problems trying to do some dat import on an intermediate schema (that later we move on to the final one). As we rely on things like extensions (for example PostGIS), the "run_insert" sql file did not fully solved the problem.
After a while, we've found that at least with Postgres 9.3 the solution is far easier... just create your SQL script always specifying the schema when refering to the table:
CREATE TABLE "my_schema"."my_table" (...);
COPY "my_schema"."my_table" (...) FROM stdin;
This way using psql -f xxxxx works perfectly, and you don't need to change search_paths nor use intermediate files (and won't hit extension schema problems).
what is wrong with this command:
pg_dump -U postgres -W admin --disable-triggers -a -t employees -f D:\ddd.txt postgres
This is giving error of too many command-line arguments
Looks like its the -W option. There is no value to go with that option.
-W, --password force password prompt (should happen automatically)
If you want to run the command without typing is a password, use a .pgpass file.
http://www.postgresql.org/docs/9.1/static/libpq-pgpass.html
For posterity, note that pg_dump and pg_restore (and many other commands) cannot process long hyphens that word processors create. If you are cut-pasting command lines from a word processor, be sure it hasn't converted your hyphens to something else in editing. Else you will get command lines that look correct but hopelessly confuse the argument parsers in these tools.
pg_dump and pg_restore need to ask password on commandline, if you put it command, they always give "too many command-line arguments" error. You can use below for setting related environment variable in commandline or batch file:
"SET PGPASSWORD=<password>"
so that you are not asked to enter password manually in your batch file. They use given environment variable.
Instead of passing password with -W flag start with setting temporary variable for postgres:
PGPASSWORD="mypass" pg_dump -U postgres--disable-triggers -a -t employees -f D:\ddd.txt postgres
-W -> will prompt for a password
to take full DB dump
use some thing like
pg_dump -h 192.168.44.200 -p 5432 -U postgres -W -c -C -Fc -f C:\MMM\backup10_3.backup DATABASE_NAME
I got this from copy-pasting, where 1 of the dashes were different.
Was: –-host= (first dash i a "long" dash)
Corrected to --host= solved it
Another option is to add ~/.pgpass file with content like this:
hostname:port:database:username:password
read more here
Additionally, if you don't want password prompt, use connection string directly.
pg_dump 'postgresql://<username>:<password>#localhost:5432/<dbname>'
So, combination with options in original question,
pg_dump 'postgresql://postgres:<password>#localhost:5432/postgres' --table='"employees"' --format='t' --file='D:\ddd.txt' --data-only --disable-triggers
(Don't forget to use quotes when you have letter-casing issues)
reference:
https://www.postgresql.org/docs/current/app-pgdump.html
Postgres dump specific table with a capital letter
2021-11-30, pg v12, windows 10
pg_dump -U postgres -W -F t postgres > C:\myfolder\pg.tar
-U "postgres" as username,
-W to prompt for psd,
-F t means format is .tar,
> C:\myfolder\pg.tar is the destination path and filename