I'm trying to pg_dump certain schemas from my Heroku-hosted PSQL database into a local file.
Heroku provides me a DATABASE_URL in the form
# postgres://username:password#host:port/database
postgres://abcde:wxyz#ec2-21-82-72-112.compute-1.amazonaws.com:5762/riza3dj029012
Based on the above I tried dumping some schemas -
> pg_dump --username=abcde --host=ec2-21-82-72-112.compute-1.amazonaws.com --port=5762 --dbname=riza3dj029012 --create --schema=my_schema --password > ~/pg_dump.dmp
> password: (enter password)
Is there any way to provide the password as a flag so I don't have to type it in manually? I want to automate this in a script.
I know --no-password exists, but not sure what use that is, because it just prevents prompting for a password and then authentication (obviously) fails.
Thanks!
As #eabates mentioned in the comments you can create a .pgpass file for this purpose. More info can be found in the official documentation for Postgres: https://www.postgresql.org/docs/9.5/static/libpq-pgpass.html
Just create a file named .pgpass in the user home directory with as many lines as needed in the following format:
hostname:port:database:username:password
Related
I want to use rundeck to allow app user to reset their own password without admin access.
So I would like to execute this type of command:
export PASSFILE=/usr/pgsql-13/.pgpass ; /usr/pgsql-13/bin/psql -h hostname -U postgres -c """alter user ${job.username} PASSWORD '${option.Password}';"""
But the script fails because postgres ask for password
Any one have a workaround? The postgres DB is not a classic DB but aws aurora DB, so I don't have access to pg_hba file
Many thanks
If you want to use interactive commands on Rundeck you can use expect command, in that way you can "wrap" your command to "answer" any interactive command "question", that was answered here.
The environment valuable is called PGPASSFILE.
An alternative and simpler way is to specify the passfile connection parameter:
/usr/pgsql-13/bin/psql 'host=hostname user=postgres passfile=/usr/pgsql-13/.pgpass' -c "alter user ${job.username} PASSWORD '${option.Password}';"
You can also use the password parameter to specify the password in the connect string.
When login to the database using psql -u postgres, all the commands entered there can be seen in and recalled from history.
I created a role with a password and I would like to clear that entry.
How can I clear the history?
Please check on the /home/user/.psql_history, then open the file if we want we can clear all the commands or the necessary commands and save the file.
Because you used -u postgres, the file should be in the home of your postgres user. In my case it is /var/lib/postgresql/.psql_history.
I'm having difficulty restoring a DB to an AWS RDS Postgresql instance. Context is that i am backing up from one RDS instance and restoring to another RDS insurance. They both have the same version of Postgresql 9.6.5.
I was able to take a dump using the following command:
./pg_dump.exe -U dbuser -W -h prod-pgsql-rds.3ft5coqxjdnq.eu-west-2.rds.amazonaws.com -d devdb > c:\tmp\backup.sql
From the resulting .sql file, I then attempted a restore to another RDS instance which is also using Postgresql 9.6.5 using below command:
./pg_restore.exe -U dbuser -d testdevdb -h dev-pgsql-rds.cym8coqx52lq.eu-west-2.rds.amazonaws.com "c:\tmp\backup.sql"
*I also tried the -f switch in the above restore command instead of the " " quotes before/after the file name
But when I try to restore it to a newly created database I get the following error:
pg_restore: [archiver] input file does not appear to be a valid archive
Can anyone help? FYI, I am using PGAdmin 4 via Windows PowerShell. I have to edit some of the values in the strings above due to data sensitivity.
pg_restore is only used for the other, non-plain-text output formats that pg_dump can output. For .sql dumps, you just use psql. See the docs on restoring from backups.
In a Unix env, you'd do psql [yourflags] < /tmp/backup.sql, but I'm unfamiliar with powershell and don't know if it supports < for input redirection; hopefully either it's present or you know the equivalent PowerShell syntax.
So I couldn't get psql or pg_restore to work so opted to import the .SQL file into via the SQL query tool in PGAmdin. This through up some errors so had to make several changes to the .SQL file and perform below:
Commented out a couple of lines that were causing errors
Elevated permissions for the user and made him the owner of for the Schema and DB properties by right-clicking on these via PGAdmin
The .sql file was making several references to the user from the source RDS DB so had to do a find and replace with a user account created for the destination RDS DB. Alternatively, I could have just created a new user on the destination DB with the same username and password as the source DB and then make him the owner in ref to step 2.
I need to run multiple commands on an AWS Postgres RDS instance I have. I don't want to enter the password each time. I'm trying to use the .pgpass file but I'm running into errors. The first time I ran into an error which said 'role "ubuntu username" does not exist'. I logged in as the postgres user and created that username. After this, the error I get said database does not exist. I have a feeling these errors have nothing to do with trying to connect to the AWS RDS instance.
psql --host=<awshost> --port=5432 --username=<awsrdsusername> --password --dbname=<dbname asks for a password and then logs me in after I enter it. Now I put a file in /home/<ubuntuusername> called .pgpass which has <awshost>:5432:<dbname>:<awsrdsusername>:<password>. Permissions for this file are set to 0600. Now when I run psql from the terminal and that produces the error - psql: FATAL: role "<ubuntuusername>" does not exist. These steps are as outlined on this page.
Can someone help me with the steps to get a pgpass file to connect to an AWS RDS instance?
.pgpass doesn't provide connection information. You seem to expect that after you create a .pgpass file, you can run psql without arguments and it'll know where to connect. That is not the case.
The hostname, port, etc you put in .pgpass are there so that PostgreSQL knows which line to look at when matching the connection info to find its password.
If you had to run:
psql -h something
to connect without a .pgpass file, you still have to run the same thing to connect with a .pgpass file.
If you run psql without arguments it'll connect to the local PostgreSQL (if any), using the current unix username as the postgresql username and the database to connect to. That's why you get the error you do.
If you want to change the default connection, you can use environment variables like PGHOST, PGPORT, etc, and/or a .pgservice.conf file.
See the manual to learn more.
I have a fabric script that dumps database on server. And I can use it on multiple servers with the PostgreSQL database. The command is simple:
sudo("su postgres -c \"PGPASSWORD=%s pg_dump %s > /tmp/telemedia_newdb\""
% (HOST_SOURCE_DB_UPASS,HOST_SOURCE_DB))
But sometimes, Postgres does not ask for a password at all ...
Will this command fail without a password prompting from Postgres? (Or I know that it will not prompt and HOST_SOURCE_DB_UPASS=''). I want THIS code to work with or without password.
It all depends on how you set up access to your database in pg_hba.conf. There is a separate config file per database cluster (effectively per port) and settings can be different from database to database.
So, yes, if you have set it up that way, then the system user postgres will have password-less access to some databases but is prompted to enter a password for others. The default is that the system user postgres has password-less access to every database as database user of the same name (postgres).
If you provide a password in the command with the environment variable PGPASSWORD, but no password is needed, it will be ignored silently.
However, I quote the manual here:
PGPASSWORD (...) Use of this environment variable is not recommended for security reasons.
You can use a password file to provide passwords automatically (.pgpass on Unix systems). pg_dump will use it.
Finally, consider the command line options:
--no-password
--password
to force pg_dump to either prompt or not prompt for a password. If a password is required but disabled by --no-password, pg_dump will fail.
I would enable password-less access for the system user postgres to every database in the config file pg_hba.conf. Use peer or ident authentication methods. Then you don't have to provide a password and the script will always work:
local all postgres ident
Your script would be simplified to (untested):
sudo("su postgres -c \"pg_dump %s > /tmp/telemedia_newdb\"" % (HOST_SOURCE_DB))