I need to download my database at heroku, how to add in these flags: -a (data only), -x (no privileges), -O (no owner) to the CLI ??
Currently I use:
heroku pgbackups:capture
$ curl -o latest.dump `heroku pgbackups:url`
It doesn't seem like you can pass flags to pgbackups:capture. You can, however, uses pg_dump directly.
pg_dump DATABASE_NAME -h DATABASE_HOST -p PORT -U USER -F c -a -x -O -f db.dump
You can get the database values by running heroku pg:credentials DATABASE_URL You can also use the plugin a colleague and I wrote: parse_db_url. This will let you run a command like heroku pg:parse_db_url --format pg_dump and get a usable pg_dump command as output.
Related
I have some .sql files with thousands of INSERT statements in them and need to run these inserts on my PostgreSQL database in order to add them to a table. The files are that large that it is impossible to open them and copy the INSERT statements into an editor window and run them there. I found on the Internet that you can use the following by navigating to the bin folder of your PostgreSQL install:
psql -d myDataBase -a -f myInsertFile
In my case:
psql -d HIGHWAYS -a -f CLUSTER_1000M.sql
I am then asked for a password for my user, but I cannot enter anything and when I hit enter I get this error:
psql: FATAL: password authentication failed for user "myUsername"
Why won't it let me enter a password. Is there a way round this as it is critical that I can run these scripts?
I got around this issue by adding a new entry in my pg_hba.conf file with the following structure:
# IPv6 local connections:
host myDbName myUserName ::1/128 trust
The pg_hba.conf file can usually be found in the 'data' folder of your PostgreSQL install.
Of course, you will get a fatal error for authenticating, because you do not include a user name...
Try this one, it is OK for me :)
psql -U username -d myDataBase -a -f myInsertFile
If the database is remote, use the same command with host
psql -h host -U username -d myDataBase -a -f myInsertFile
You should do it like this:
\i path_to_sql_file
See:
You have four choices to supply a password:
Set the PGPASSWORD environment variable. For details see the manual: http://www.postgresql.org/docs/current/static/libpq-envars.html
Use a .pgpass file to store the password. For details see the manual: http://www.postgresql.org/docs/current/static/libpq-pgpass.html
Use "trust authentication" for that specific user: http://www.postgresql.org/docs/current/static/auth-methods.html#AUTH-TRUST
Since PostgreSQL 9.1 you can also use a connection string: https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
Use this to execute *.sql files when the PostgreSQL server is located in a difference place:
psql -h localhost -d userstoreis -U admin -p 5432 -a -q -f /home/jobs/Desktop/resources/postgresql.sql
-h PostgreSQL server IP address
-d database name
-U user name
-p port which PostgreSQL server is listening on
-f path to SQL script
-a all echo
-q quiet
Then you are prompted to enter the password of the user.
EDIT: updated based on the comment provided by #zwacky
If you are logged in into psql on the Linux shell the command is:
\i fileName.sql
for an absolute path and
\ir filename.sql
for the relative path from where you have called psql.
export PGPASSWORD=<password>
psql -h <host> -d <database> -U <user_name> -p <port> -a -w -f <file>.sql
Via the terminal log on to your database and try this:
database-# >#pathof_mysqlfile.sql
or
database-#>-i pathof_mysqlfile.sql
or
database-#>-c pathof_mysqlfile.sql
You can give both user name and PASSSWORD on the command line itself with the "-d" parameter
psql -d "dbname='urDbName' user='yourUserName' password='yourPasswd' host='yourHost'" -f yourFileName.sql
you could even do it in this way:
sudo -u postgres psql -d myDataBase -a -f myInsertFile
If you have sudo access on machine and it's not recommended for production scripts just for test on your own machine it's the easiest way.
2021 Solution
if your PostgreSQL database is on your system locally.
psql dbname < sqldump.sql username
If its hosted online
psql -h hostname dbname < sqldump.sql username
If you have any doubts or questions, please ask them in the comments.
Walk through on how to run an SQL on the command line for PostgreSQL in Linux:
Open a terminal and make sure you can run the psql command:
psql --version
which psql
Mine is version 9.1.6 located in /bin/psql.
Create a plain textfile called mysqlfile.sql
Edit that file, put a single line in there:
select * from mytable;
Run this command on commandline (substituting your username and the name of your database for pgadmin and kurz_prod):
psql -U pgadmin -d kurz_prod -a -f mysqlfile.sql
The following is the result I get on the terminal (I am not prompted for a password):
select * from mytable;
test1
--------
hi
me too
(2 rows)
psql -h localhost -d userstoreis -U admin -p 5432 -a -q -f /home/jobs/Desktop/resources/postgresql.sql
Parameter explanations:
-h PostgreSQL server IP address
-d database name
-U user name
-p port which PostgreSQL server is listening on
-f path to SQL script
-a all echo
-q quiet
You can open a command prompt and run as administrator. Then type
../bin>psql -f c:/...-h localhost -p 5432 -d databasename -U "postgres"
Password for user postgres: will show up.
Type your password and enter. I couldn't see the password what I was typing, but this time when I press enter it worked. Actually I was loading data into the database.
I achived that wrote (located in the directory where my script is)
::someguy#host::$sudo -u user psql -d my_database -a -f file.sql
where -u user is the role who owns the database where I want to execute the script then the psql connects to the psql console after that -d my_database loads me in mydatabase finally -a -f file.sql where -a echo all input from the script and -f execute commands from file.sql into mydatabase, then exit.
I'm using:
psql (PostgreSQL) 10.12
on (Ubuntu 10.12-0ubuntu0.18.04.1)
A small improvement in #wingman__7 's 2021 answer: if your username contains certain characters (an underscore in my case), you need to pass it with the -U flag.
This worked for me:
$ psql -h db.host -d db_name -U my_user < query.sql
Try using the following command in the command line console:
psql -h localhost -U postgres -f restore.sql
I'm trying to copy a database from one remote server to another one. I've tried several different commands from my terminal (macOS):
"pg_dump -U postgres -d [DB] -f [DB].sql"
"pg_dump -U postgres -d [DB] -h [Host] -f [DB].sql"
But nothing works. I get errors like "pg_dump: error: connection to database [DB] failed: FATAL: database [DB] does not exist".
Any ideas how to solve this problem? I've tried to edit the pg_hba.conf, but it didn't work as well..
The procedure to make this work is :
First sub-option below outputs plain text file, second custom format(binary) file
a) pg_dump -C -h host_name -U user_name -d database_name -f database.sql
-C tells pg_dump to provide the command to create the database
on restore.
b) pg_dump -Fc -h host_name -U user_name -d database_name-f database.out
To restore you need different programs
a) For plain text option 1a do:
psql -d postgres -h host_name -U user_name -f database.sql
You need to connect to existing database, postgres in this case,
and then the commands from -C above will create
the database(database_name) and then connect to it for rest of operation.
b) For custom format option 1b:
pg_restore -C -d postgres -h host_name -U user_name database.out
Note: just specify the dump file(database.out) do not use -f.
More options and details can be found:
pg_dump
and
pg_restore
Look in the Notes section at the bottom of link for examples.
I have exported a schema from my PostgreSQL database (v10.6) and it is 634Mb using the following command:
pg_dump -Fc -n schema -v -O -U username -p 5432 -h host -d database > export.dump.gz
Using the exact same export command from v12 is 1.3Gb. I have read the pg_dump docs for v12 and v10 and there doesn't seem to have been a change in compression but checked and used the following command with the extra compression switch Z and the default value of 6.
pg_dump -Fc -n schema -v -O -U username -p 5432 -h host -d database -Z 6 > export.dump.gz
And this made no difference.
Does anyone know if the pg_dump was changed for v12?
I would like to have a way to enter into the Postgresql container and get a data dump from it.
Use the following command from a UNIX or a Windows terminal:
docker exec <container_name> pg_dump <schema_name> > backup
The following command will dump only inserts from all tables:
docker exec <container_name> pg_dump --column-inserts --data-only <schema_name> > inserts.sql
I have container named postgres with mounted volume -v /backups:/backups
To backup gziped DB my_db I use:
docker exec postgres pg_dump -U postgres -F t my_db | gzip >/backups/my_db-$(date +%Y-%m-%d).tar.gz
Now I have
user#my-server:/backups$ ls
my_db-2016-11-30.tar.gz
Although the mountpoint solution above looked promising, the following is the only solution that worked for me after multiple iterations:
docker run -it -e PGPASSWORD=my_password postgres:alpine pg_dump -h hostname -U my_user my_db > backup.sql
What was unique in my case: I have a password on the database that needs to be passed in; needed to pass in the tag (alpine); and finally the hosts version of the psql tools were different to the docker versions.
This one, using container_name instead of database_scheme's one, works for me:
docker exec {container_name} pg_dump -U {user_name} > {backup_file_name}
In instance, for me, database name, user and password are supposed declared in docker-compose.yaml
I wish it could help someone
for those who suffered with permissions, I used this following command with success to perform my dump:
docker exec -i MY_CONTAINER_NAME /bin/bash -c "PGPASSWORD=MY_PASSWORD pg_dump -Fc -h localhost -U postgres MY_DB_NAME" > /home/MY_USER/db-$(date +%d-%m-%y).backup
This will mount the pwd and include your environment variables
docker run -it --rm \
--env-file <(env) \
-w /working \
--volume $(pwd):/working \
postgres:latest /usr/bin/pg_dump -Fc -h localhost -U postgres MY_DB_NAME" > /working/db-$(date +%d-%m-%y).backup
Another workaround method is to start postgre sql with a mountpoint to the location of the dump in docker.
like docker run -v <location of the files>.
Then perform a docker inspect on the docker running container
docker inspect <image_id>
you can find "Volumes" tag inside and a corresponding location.Go to the location and you can find all the postgresql/mysql files.It worked for me.Let us know if that worked for you also.
Good luck
To run the container that has the Postgres user and password, you need to have preconfigured variables as container environment variable.
For example:
docker run -it --rm --link <container_name>:<data_container_name> -e POSTGRES_PASSWORD=<password> postgres /usr/bin/pg_dump -h <data_container_name> -d <database_name> -U <postgres_username> > dump.sql
I have some .sql files with thousands of INSERT statements in them and need to run these inserts on my PostgreSQL database in order to add them to a table. The files are that large that it is impossible to open them and copy the INSERT statements into an editor window and run them there. I found on the Internet that you can use the following by navigating to the bin folder of your PostgreSQL install:
psql -d myDataBase -a -f myInsertFile
In my case:
psql -d HIGHWAYS -a -f CLUSTER_1000M.sql
I am then asked for a password for my user, but I cannot enter anything and when I hit enter I get this error:
psql: FATAL: password authentication failed for user "myUsername"
Why won't it let me enter a password. Is there a way round this as it is critical that I can run these scripts?
I got around this issue by adding a new entry in my pg_hba.conf file with the following structure:
# IPv6 local connections:
host myDbName myUserName ::1/128 trust
The pg_hba.conf file can usually be found in the 'data' folder of your PostgreSQL install.
Of course, you will get a fatal error for authenticating, because you do not include a user name...
Try this one, it is OK for me :)
psql -U username -d myDataBase -a -f myInsertFile
If the database is remote, use the same command with host
psql -h host -U username -d myDataBase -a -f myInsertFile
You should do it like this:
\i path_to_sql_file
See:
You have four choices to supply a password:
Set the PGPASSWORD environment variable. For details see the manual: http://www.postgresql.org/docs/current/static/libpq-envars.html
Use a .pgpass file to store the password. For details see the manual: http://www.postgresql.org/docs/current/static/libpq-pgpass.html
Use "trust authentication" for that specific user: http://www.postgresql.org/docs/current/static/auth-methods.html#AUTH-TRUST
Since PostgreSQL 9.1 you can also use a connection string: https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
Use this to execute *.sql files when the PostgreSQL server is located in a difference place:
psql -h localhost -d userstoreis -U admin -p 5432 -a -q -f /home/jobs/Desktop/resources/postgresql.sql
-h PostgreSQL server IP address
-d database name
-U user name
-p port which PostgreSQL server is listening on
-f path to SQL script
-a all echo
-q quiet
Then you are prompted to enter the password of the user.
EDIT: updated based on the comment provided by #zwacky
If you are logged in into psql on the Linux shell the command is:
\i fileName.sql
for an absolute path and
\ir filename.sql
for the relative path from where you have called psql.
export PGPASSWORD=<password>
psql -h <host> -d <database> -U <user_name> -p <port> -a -w -f <file>.sql
Via the terminal log on to your database and try this:
database-# >#pathof_mysqlfile.sql
or
database-#>-i pathof_mysqlfile.sql
or
database-#>-c pathof_mysqlfile.sql
You can give both user name and PASSSWORD on the command line itself with the "-d" parameter
psql -d "dbname='urDbName' user='yourUserName' password='yourPasswd' host='yourHost'" -f yourFileName.sql
you could even do it in this way:
sudo -u postgres psql -d myDataBase -a -f myInsertFile
If you have sudo access on machine and it's not recommended for production scripts just for test on your own machine it's the easiest way.
2021 Solution
if your PostgreSQL database is on your system locally.
psql dbname < sqldump.sql username
If its hosted online
psql -h hostname dbname < sqldump.sql username
If you have any doubts or questions, please ask them in the comments.
Walk through on how to run an SQL on the command line for PostgreSQL in Linux:
Open a terminal and make sure you can run the psql command:
psql --version
which psql
Mine is version 9.1.6 located in /bin/psql.
Create a plain textfile called mysqlfile.sql
Edit that file, put a single line in there:
select * from mytable;
Run this command on commandline (substituting your username and the name of your database for pgadmin and kurz_prod):
psql -U pgadmin -d kurz_prod -a -f mysqlfile.sql
The following is the result I get on the terminal (I am not prompted for a password):
select * from mytable;
test1
--------
hi
me too
(2 rows)
psql -h localhost -d userstoreis -U admin -p 5432 -a -q -f /home/jobs/Desktop/resources/postgresql.sql
Parameter explanations:
-h PostgreSQL server IP address
-d database name
-U user name
-p port which PostgreSQL server is listening on
-f path to SQL script
-a all echo
-q quiet
You can open a command prompt and run as administrator. Then type
../bin>psql -f c:/...-h localhost -p 5432 -d databasename -U "postgres"
Password for user postgres: will show up.
Type your password and enter. I couldn't see the password what I was typing, but this time when I press enter it worked. Actually I was loading data into the database.
I achived that wrote (located in the directory where my script is)
::someguy#host::$sudo -u user psql -d my_database -a -f file.sql
where -u user is the role who owns the database where I want to execute the script then the psql connects to the psql console after that -d my_database loads me in mydatabase finally -a -f file.sql where -a echo all input from the script and -f execute commands from file.sql into mydatabase, then exit.
I'm using:
psql (PostgreSQL) 10.12
on (Ubuntu 10.12-0ubuntu0.18.04.1)
A small improvement in #wingman__7 's 2021 answer: if your username contains certain characters (an underscore in my case), you need to pass it with the -U flag.
This worked for me:
$ psql -h db.host -d db_name -U my_user < query.sql
Try using the following command in the command line console:
psql -h localhost -U postgres -f restore.sql