I'm trying to restore my Postgres database to an RDS Postgres Instance in the AWS using pg restore from an EC2 instance.
I am using the following command to restore the database:
pg_restore -v -h host --no-owner -d PostgresDB postgres.dump
Now the problem is that originally, I didn't specify the --no-owner option, and since the owner of the local database that has been backed up, and the owner of the RDS Instance aren't the same. This threw an error, which is why I read that specifying this option helps solve the issue.
However, now I get a
pg_restore: [archiver (db)] connection to database "XY" failed:
FATAL: password authentication failed for user "ec2-user"
error message, although the password is right. Now I read that this does happens with Postgresql, but I can't find a way to solve this on EC2. According to this thread, I need to change the format of one of the configuration files. But this my postgres is an AWS Instance, how can I achieve this? I browsed my EC2 server instance and didn't find any pgpass.conf file (according to 'which' the file doesn't exist on the server). How can I solve this?
What username did you create for Postgres? Surely it isn't ec2-user. You need to be specifying the username and password with the --username and --password options. Here is the documentation.
Related
I'm having trouble performing the restore from a dump. The scenario is as follows: I am migrating an environment from GCP to AWS, and at the moment I am working on the migration of the bank.
A partner dumped db that is in GCP and placed the file on AWS S3 (I don't know the command he used to perform the dump).
I created an EC2 in the AWS environment and copied the dump from S3 to EC2 (the file is 13 GB). I also created the RDS to host the new db with all the correct security group settings.
Here comes the problem, I connect to the RDS from the server (EC2) without problems, but when doing the restore using pg_restore I get the following error message: pg_restore: too many command line arguments (first is "dbclient. dump ").
The complete command I used was this:
pg_restore -h client-aurora-cluster-hmg-legado-instance-1.c23ltjbbz7ms.us-east-1.rds.amazonaws.com -U postgres -d db_hmg_legado dbclient.dump -W
OK, I changed the approach. I tried with psql instead of pg_restore and then the command was like this:
psql -h client-aurora-cluster-hmg-legado-instance-1.c23ltjbbz7ms.us-east-1.rds.amazonaws.com -U postgres -d db_hmg_legado dbclient.dump -W
Only this time it worked !!!!
But I received some error messages while performing the restore. Which I put below:
psql: dbclient.dump: 23: ERROR: schema "dw" already exists
CREATE EXTENSION
psql: dbclient.dump: 37: ERROR: must be owner of extension hstore
CREATE EXTENSION
psql: dbclient.dump: 51: ERROR: must be owner of extension intarray
CREATE EXTENSION
psql: dbclient.dump: 65: ERROR: must be owner of extension pg_trgm
CREATE EXTENSION
psql: dbclient.dump: 79: ERROR: must be owner of extension unaccent
But the restore takes a long time and is partially finished.
In general I wanted to understand why pg_restore didn't work. Has anyone ever experienced this?
And about these owner errors does anyone know how to resolve this using psql?
As documented in the manual the file to be restored is the last parameter and it is specified without a "switch". But you are using -W after the dump file. Move the -W parameter somewhere before that (although it's usually not necessary to begin with)
So you need something like this:
pg_restore -W -h ... -U postgres -d db_hmg_legado dbclient.dump
However, if the restore worked when using psql then the dump file is a "plain text" dump which can't be restored using pg_restore to begin with.
Concerning the errors:
You should restore the dump into an empty database that doesn't contain any schemas except the default ones.
You need a superuser for CREATE EXTENSION, which you don't have in a hosted database. So pre-install these extensions with the techniques that Amazon provides, then restore the dump and ignore the errors.
I'm having difficulty restoring a DB to an AWS RDS Postgresql instance. Context is that i am backing up from one RDS instance and restoring to another RDS insurance. They both have the same version of Postgresql 9.6.5.
I was able to take a dump using the following command:
./pg_dump.exe -U dbuser -W -h prod-pgsql-rds.3ft5coqxjdnq.eu-west-2.rds.amazonaws.com -d devdb > c:\tmp\backup.sql
From the resulting .sql file, I then attempted a restore to another RDS instance which is also using Postgresql 9.6.5 using below command:
./pg_restore.exe -U dbuser -d testdevdb -h dev-pgsql-rds.cym8coqx52lq.eu-west-2.rds.amazonaws.com "c:\tmp\backup.sql"
*I also tried the -f switch in the above restore command instead of the " " quotes before/after the file name
But when I try to restore it to a newly created database I get the following error:
pg_restore: [archiver] input file does not appear to be a valid archive
Can anyone help? FYI, I am using PGAdmin 4 via Windows PowerShell. I have to edit some of the values in the strings above due to data sensitivity.
pg_restore is only used for the other, non-plain-text output formats that pg_dump can output. For .sql dumps, you just use psql. See the docs on restoring from backups.
In a Unix env, you'd do psql [yourflags] < /tmp/backup.sql, but I'm unfamiliar with powershell and don't know if it supports < for input redirection; hopefully either it's present or you know the equivalent PowerShell syntax.
So I couldn't get psql or pg_restore to work so opted to import the .SQL file into via the SQL query tool in PGAmdin. This through up some errors so had to make several changes to the .SQL file and perform below:
Commented out a couple of lines that were causing errors
Elevated permissions for the user and made him the owner of for the Schema and DB properties by right-clicking on these via PGAdmin
The .sql file was making several references to the user from the source RDS DB so had to do a find and replace with a user account created for the destination RDS DB. Alternatively, I could have just created a new user on the destination DB with the same username and password as the source DB and then make him the owner in ref to step 2.
I have setup PostgreSQL hot stand by replication on Ubuntu. I need to know if master DB server is down, then how to get the backup from the slave.
I have tried this command
pg_dump testdb > /var/lib/postgresql/20190306.bak -p 5433
I got this error:
pg_dump: [archiver (db)] connection to database "channeldb" failed:
FATAL: role "root" does not exist
This specific error has nothing to do with this being a standby server.
Rather, you forgot to use the -U option to specify the database user, so pg_dump assumes it is the same as the operating system user.
Don't use the root user for anything but administrative activities!
I need to run multiple commands on an AWS Postgres RDS instance I have. I don't want to enter the password each time. I'm trying to use the .pgpass file but I'm running into errors. The first time I ran into an error which said 'role "ubuntu username" does not exist'. I logged in as the postgres user and created that username. After this, the error I get said database does not exist. I have a feeling these errors have nothing to do with trying to connect to the AWS RDS instance.
psql --host=<awshost> --port=5432 --username=<awsrdsusername> --password --dbname=<dbname asks for a password and then logs me in after I enter it. Now I put a file in /home/<ubuntuusername> called .pgpass which has <awshost>:5432:<dbname>:<awsrdsusername>:<password>. Permissions for this file are set to 0600. Now when I run psql from the terminal and that produces the error - psql: FATAL: role "<ubuntuusername>" does not exist. These steps are as outlined on this page.
Can someone help me with the steps to get a pgpass file to connect to an AWS RDS instance?
.pgpass doesn't provide connection information. You seem to expect that after you create a .pgpass file, you can run psql without arguments and it'll know where to connect. That is not the case.
The hostname, port, etc you put in .pgpass are there so that PostgreSQL knows which line to look at when matching the connection info to find its password.
If you had to run:
psql -h something
to connect without a .pgpass file, you still have to run the same thing to connect with a .pgpass file.
If you run psql without arguments it'll connect to the local PostgreSQL (if any), using the current unix username as the postgresql username and the database to connect to. That's why you get the error you do.
If you want to change the default connection, you can use environment variables like PGHOST, PGPORT, etc, and/or a .pgservice.conf file.
See the manual to learn more.
Well I installed the latest postgreql database on my Windows 7.
Now I'm trying to create a database via the psql.exe command line
When I open it, it says
psql: FATAL: database "Jansu" does not exist
So I read somewhere, that when no database is specified, it tried to find database with my username or something.
Anyways..how do i create a new database, when I can't access the commandline.
Read psql syntax. You can specify database, user and other parameters. If it's a new installation, there should be a default database 'postgres', you can connect to that one.
psql -U postgres postgres
(In Unix environments you might need to add -h localhost in order to force a TCP connection, otherwise it'd try to use Unix-domain sockets, which might not work for other than the postgres user. )
You can create databases from there, or from the command line with createdb