I have developed a shell script whose job is to take the dump of postgres DB. Below is the snippet:
#!/bin/sh
today=$(date +"%Y-%m-%d")
yes "password" | sudo -S sudo su - postgres <<EOF
/usr/pgsql-11/bin/pg_dump -U postgres -d db_name > /home/db_backup/db_name_$today.sql
EOF
exit
However, this script is NOT running because of the below reason:
[sudo] password for user: Sorry, Try again
However, when I use sudo su - postgres and then provide password, it is working as expected. And interestingly, if now I run the above shell script after the login, it runs absolutely fine.
What I am missing here.
It is dangerous to store passwords in scripts, so please do not do it.
Modify your /etc/sudoers file by running sudo visudo and adding a line like this at the bottom:
%sudo ALL=(postgres) NOPASSWD: /usr/bin/psql
This allows anyone with sudo permission to run /usr/bin/psql to postgres on any host (ALL) with no password.
Now your script should work this way:
#!/bin/sh
today=$(date +"%Y-%m-%d")
sudo -b -n -H -u postgres /usr/pgsql-11/bin/pg_dump -U postgres -d db_name > /home/db_backup/db_name_$today.sql
Make sure postgres can write to the directory /home/db_backup/.
Related
I have some .sql files with thousands of INSERT statements in them and need to run these inserts on my PostgreSQL database in order to add them to a table. The files are that large that it is impossible to open them and copy the INSERT statements into an editor window and run them there. I found on the Internet that you can use the following by navigating to the bin folder of your PostgreSQL install:
psql -d myDataBase -a -f myInsertFile
In my case:
psql -d HIGHWAYS -a -f CLUSTER_1000M.sql
I am then asked for a password for my user, but I cannot enter anything and when I hit enter I get this error:
psql: FATAL: password authentication failed for user "myUsername"
Why won't it let me enter a password. Is there a way round this as it is critical that I can run these scripts?
I got around this issue by adding a new entry in my pg_hba.conf file with the following structure:
# IPv6 local connections:
host myDbName myUserName ::1/128 trust
The pg_hba.conf file can usually be found in the 'data' folder of your PostgreSQL install.
Of course, you will get a fatal error for authenticating, because you do not include a user name...
Try this one, it is OK for me :)
psql -U username -d myDataBase -a -f myInsertFile
If the database is remote, use the same command with host
psql -h host -U username -d myDataBase -a -f myInsertFile
You should do it like this:
\i path_to_sql_file
See:
You have four choices to supply a password:
Set the PGPASSWORD environment variable. For details see the manual: http://www.postgresql.org/docs/current/static/libpq-envars.html
Use a .pgpass file to store the password. For details see the manual: http://www.postgresql.org/docs/current/static/libpq-pgpass.html
Use "trust authentication" for that specific user: http://www.postgresql.org/docs/current/static/auth-methods.html#AUTH-TRUST
Since PostgreSQL 9.1 you can also use a connection string: https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
Use this to execute *.sql files when the PostgreSQL server is located in a difference place:
psql -h localhost -d userstoreis -U admin -p 5432 -a -q -f /home/jobs/Desktop/resources/postgresql.sql
-h PostgreSQL server IP address
-d database name
-U user name
-p port which PostgreSQL server is listening on
-f path to SQL script
-a all echo
-q quiet
Then you are prompted to enter the password of the user.
EDIT: updated based on the comment provided by #zwacky
If you are logged in into psql on the Linux shell the command is:
\i fileName.sql
for an absolute path and
\ir filename.sql
for the relative path from where you have called psql.
export PGPASSWORD=<password>
psql -h <host> -d <database> -U <user_name> -p <port> -a -w -f <file>.sql
Via the terminal log on to your database and try this:
database-# >#pathof_mysqlfile.sql
or
database-#>-i pathof_mysqlfile.sql
or
database-#>-c pathof_mysqlfile.sql
You can give both user name and PASSSWORD on the command line itself with the "-d" parameter
psql -d "dbname='urDbName' user='yourUserName' password='yourPasswd' host='yourHost'" -f yourFileName.sql
you could even do it in this way:
sudo -u postgres psql -d myDataBase -a -f myInsertFile
If you have sudo access on machine and it's not recommended for production scripts just for test on your own machine it's the easiest way.
2021 Solution
if your PostgreSQL database is on your system locally.
psql dbname < sqldump.sql username
If its hosted online
psql -h hostname dbname < sqldump.sql username
If you have any doubts or questions, please ask them in the comments.
Walk through on how to run an SQL on the command line for PostgreSQL in Linux:
Open a terminal and make sure you can run the psql command:
psql --version
which psql
Mine is version 9.1.6 located in /bin/psql.
Create a plain textfile called mysqlfile.sql
Edit that file, put a single line in there:
select * from mytable;
Run this command on commandline (substituting your username and the name of your database for pgadmin and kurz_prod):
psql -U pgadmin -d kurz_prod -a -f mysqlfile.sql
The following is the result I get on the terminal (I am not prompted for a password):
select * from mytable;
test1
--------
hi
me too
(2 rows)
psql -h localhost -d userstoreis -U admin -p 5432 -a -q -f /home/jobs/Desktop/resources/postgresql.sql
Parameter explanations:
-h PostgreSQL server IP address
-d database name
-U user name
-p port which PostgreSQL server is listening on
-f path to SQL script
-a all echo
-q quiet
You can open a command prompt and run as administrator. Then type
../bin>psql -f c:/...-h localhost -p 5432 -d databasename -U "postgres"
Password for user postgres: will show up.
Type your password and enter. I couldn't see the password what I was typing, but this time when I press enter it worked. Actually I was loading data into the database.
I achived that wrote (located in the directory where my script is)
::someguy#host::$sudo -u user psql -d my_database -a -f file.sql
where -u user is the role who owns the database where I want to execute the script then the psql connects to the psql console after that -d my_database loads me in mydatabase finally -a -f file.sql where -a echo all input from the script and -f execute commands from file.sql into mydatabase, then exit.
I'm using:
psql (PostgreSQL) 10.12
on (Ubuntu 10.12-0ubuntu0.18.04.1)
A small improvement in #wingman__7 's 2021 answer: if your username contains certain characters (an underscore in my case), you need to pass it with the -U flag.
This worked for me:
$ psql -h db.host -d db_name -U my_user < query.sql
Try using the following command in the command line console:
psql -h localhost -U postgres -f restore.sql
I have postgres installed on an ubuntu machine, and I am able to enter into the command line via something along the lines of:
$ sudo -u postgres psql
psql (10.15 (Ubuntu 10.15-0ubuntu0.18.04.1))
Type "help" for help.
postgres=#
And I can start/stop the server by doing something like:
$ sudo service postgresql
Usage: /etc/init.d/postgresql {start|stop|restart|reload|force-reload|status} [version ..]
Those both seem fine. However, I would like to run postgres in single-user mode to do a couple tests. On the postgres page it gives a few examples, such as:
To start a single-user mode server, use a command like
postgres --single -D /usr/local/pgsql/data other-options my_database
However, if I use the 'postgres' command, I just get an error saying I don't have that command:
$ postgres
Command 'postgres' not found, did you mean:
What do I need to install to run the 'postgres' command in order to enter single-user mode?
as you have not export the binary path that's why it's can't find your binary of postgres.
use this command:
/usr/lib/postgresql/10/bin/postgres --single -D /usr/local/pgsql/data other-options my_database
or,
you can export the path in bash
first open the bashrc with this command:nano ~/.bashrc
add this line in the end :PATH="/usr/lib/postgresql/10/bin/:$PATH"
run this command source ~/.bashrc
the just use postgres --single -D /usr/local/pgsql/data other-options my_database
you can also find where your binary is with this command : find /usr/lib -iname 'postgres'
It is already installed, it is just not in your PATH, as it is not anticipated you would use it manually.
It is probably somewhere like "/usr/lib/postgresql/10/bin/postgres", or you can use locate or find to find it.
Ubuntu has conf files spread over several places so:
/usr/lib/postgresql/13/bin/postgres --single -D /var/lib/postgresql/13/main -c "config_file=/etc/postgresql/13/main/postgresql.conf"
I try to set up an PostgreSQL slave using Docker and a bash script (I use Coreos). I have not found any way to supply a valid .pgpass.
I know I could create a PGPASSWORD environment variable, but do not wish to do so for security reasons (as stated here, http://www.postgresql.org/docs/current/static/libpq-envars.html),, and because this password should be accessible every time the recovery.conf file is used (for the primary_conninfo variable).
Dockerfile
# ...
# apt-get installs and other config
# ...
USER postgres
# Create role and db
RUN /etc/init.d/postgresql start &&\
psql --command "CREATE USER replicator WITH ENCRYPTED PASSWORD 'THEPASSWORD';" &&\
psql --command "CREATE DATABASE db WITH OWNER replicator;"
# Set the pg_pass to allow connection to master
ADD ./pgpass.conf /home/postgres/.pgpass # pgpass.conf comes my root git folder
USER root
RUN chmod 0600 /home/postgres/.pgpass
In my bash file
# ...
pg_basebackup -h host.of.master.ip -D /var/pgbackup/backup_data -U replicator -v -P
# ...
The problems seems to be that the pgpass file is not read. It seems I should use the password of the user I'm sudoing to (https://serverfault.com/questions/526170/psql-fe-sendauth-no-password-supplied), but in this case the replicator role is naturally not an available bash user. (Note that neither copying the pgpass to /home/root not /home/postgres works).
Note: my pgpass file and by remote database conf
# pgpass.conf
host.of.master.ip:5432:replication:replicator:THEPASSWORD
host.of.master.ip:5432:*:replicator:THEPASSWORD
# pg_hba.conf
host replication replicator host.of.slave.ip/24 md5
You have to create a .pgpass on the home folder of the user who's going to be running the commands (in this case, postgres). Each line of the file has to be in the format hostname:port:database:username:password and supports wildcards, so you can just set the database to "*" for example.
In my case, I have something like this...
$ sudo echo "${PRIMARY_IP}:5432:*:${REPL_USER}:${REPL_PASS}" > /var/lib/postgresql/.pgpass
$ sudo chown postgres:postgres /var/lib/postgresql/.pgpass
$ sudo chmod 0600 /var/lib/postgresql/.pgpass
$ sudo -u postgres pg_basebackup -h $PRIMARY_IP -D /var/lib/postgresql/9.4/main -U ${REPL_USER} -v -P --xlog-method=stream
Those variables (e.g. PRIMARY_IP) are set when I run the docker container with -e PRIMARY_IP=x.x.x.x
I am using Ubuntu 12.04 and Postgress 9.2.
I need to create this user with this password e.g.
postgres://admin:test101#127.0.0.1:5432
How to do that from the command line? I need to automate with a bash script. I have a fresh install.
This will create user admin with password test101 on localhost:
psql -c "CREATE USER admin WITH PASSWORD 'test101';"
To run it from any user just add the sudo -u postgres to it:
sudo -u postgres bash -c "psql -c \"CREATE USER vagrant WITH PASSWORD 'vagrant';\""
To run it on the original docker image for example you can use the su -c command with the postgres user in combination with psql:
su -c "psql -c \"CREATE ROLE my_user WITH LOGIN PASSWORD 'my_password' \"" postgres
I have some .sql files with thousands of INSERT statements in them and need to run these inserts on my PostgreSQL database in order to add them to a table. The files are that large that it is impossible to open them and copy the INSERT statements into an editor window and run them there. I found on the Internet that you can use the following by navigating to the bin folder of your PostgreSQL install:
psql -d myDataBase -a -f myInsertFile
In my case:
psql -d HIGHWAYS -a -f CLUSTER_1000M.sql
I am then asked for a password for my user, but I cannot enter anything and when I hit enter I get this error:
psql: FATAL: password authentication failed for user "myUsername"
Why won't it let me enter a password. Is there a way round this as it is critical that I can run these scripts?
I got around this issue by adding a new entry in my pg_hba.conf file with the following structure:
# IPv6 local connections:
host myDbName myUserName ::1/128 trust
The pg_hba.conf file can usually be found in the 'data' folder of your PostgreSQL install.
Of course, you will get a fatal error for authenticating, because you do not include a user name...
Try this one, it is OK for me :)
psql -U username -d myDataBase -a -f myInsertFile
If the database is remote, use the same command with host
psql -h host -U username -d myDataBase -a -f myInsertFile
You should do it like this:
\i path_to_sql_file
See:
You have four choices to supply a password:
Set the PGPASSWORD environment variable. For details see the manual: http://www.postgresql.org/docs/current/static/libpq-envars.html
Use a .pgpass file to store the password. For details see the manual: http://www.postgresql.org/docs/current/static/libpq-pgpass.html
Use "trust authentication" for that specific user: http://www.postgresql.org/docs/current/static/auth-methods.html#AUTH-TRUST
Since PostgreSQL 9.1 you can also use a connection string: https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
Use this to execute *.sql files when the PostgreSQL server is located in a difference place:
psql -h localhost -d userstoreis -U admin -p 5432 -a -q -f /home/jobs/Desktop/resources/postgresql.sql
-h PostgreSQL server IP address
-d database name
-U user name
-p port which PostgreSQL server is listening on
-f path to SQL script
-a all echo
-q quiet
Then you are prompted to enter the password of the user.
EDIT: updated based on the comment provided by #zwacky
If you are logged in into psql on the Linux shell the command is:
\i fileName.sql
for an absolute path and
\ir filename.sql
for the relative path from where you have called psql.
export PGPASSWORD=<password>
psql -h <host> -d <database> -U <user_name> -p <port> -a -w -f <file>.sql
Via the terminal log on to your database and try this:
database-# >#pathof_mysqlfile.sql
or
database-#>-i pathof_mysqlfile.sql
or
database-#>-c pathof_mysqlfile.sql
You can give both user name and PASSSWORD on the command line itself with the "-d" parameter
psql -d "dbname='urDbName' user='yourUserName' password='yourPasswd' host='yourHost'" -f yourFileName.sql
you could even do it in this way:
sudo -u postgres psql -d myDataBase -a -f myInsertFile
If you have sudo access on machine and it's not recommended for production scripts just for test on your own machine it's the easiest way.
2021 Solution
if your PostgreSQL database is on your system locally.
psql dbname < sqldump.sql username
If its hosted online
psql -h hostname dbname < sqldump.sql username
If you have any doubts or questions, please ask them in the comments.
Walk through on how to run an SQL on the command line for PostgreSQL in Linux:
Open a terminal and make sure you can run the psql command:
psql --version
which psql
Mine is version 9.1.6 located in /bin/psql.
Create a plain textfile called mysqlfile.sql
Edit that file, put a single line in there:
select * from mytable;
Run this command on commandline (substituting your username and the name of your database for pgadmin and kurz_prod):
psql -U pgadmin -d kurz_prod -a -f mysqlfile.sql
The following is the result I get on the terminal (I am not prompted for a password):
select * from mytable;
test1
--------
hi
me too
(2 rows)
psql -h localhost -d userstoreis -U admin -p 5432 -a -q -f /home/jobs/Desktop/resources/postgresql.sql
Parameter explanations:
-h PostgreSQL server IP address
-d database name
-U user name
-p port which PostgreSQL server is listening on
-f path to SQL script
-a all echo
-q quiet
You can open a command prompt and run as administrator. Then type
../bin>psql -f c:/...-h localhost -p 5432 -d databasename -U "postgres"
Password for user postgres: will show up.
Type your password and enter. I couldn't see the password what I was typing, but this time when I press enter it worked. Actually I was loading data into the database.
I achived that wrote (located in the directory where my script is)
::someguy#host::$sudo -u user psql -d my_database -a -f file.sql
where -u user is the role who owns the database where I want to execute the script then the psql connects to the psql console after that -d my_database loads me in mydatabase finally -a -f file.sql where -a echo all input from the script and -f execute commands from file.sql into mydatabase, then exit.
I'm using:
psql (PostgreSQL) 10.12
on (Ubuntu 10.12-0ubuntu0.18.04.1)
A small improvement in #wingman__7 's 2021 answer: if your username contains certain characters (an underscore in my case), you need to pass it with the -U flag.
This worked for me:
$ psql -h db.host -d db_name -U my_user < query.sql
Try using the following command in the command line console:
psql -h localhost -U postgres -f restore.sql